-- Logs begin at Thu 2022-05-19 21:08:55 UTC, end at Thu 2022-05-19 22:57:46 UTC. -- May 19 21:46:29.788195 dv-cinder-842633-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 21:46:29.796216 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 21:46:29 2022] *** May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: nodename: dv-cinder-842633-1-iscsi May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: machine: x86_64 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: clock source: unix May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: pcre jit disabled May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: detected number of CPU cores: 4 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: current working directory: / May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: detected binary path: /usr/bin/uwsgi-core May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: your processes number limit is 47680 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: your memory page size is 4096 bytes May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: detected max file descriptor number: 2048 May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: lock engine: pthread robust mutexes May 19 21:46:29.799572 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: thunder lock: enabled May 19 21:46:29.800345 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 21:46:29.800345 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 21:46:29.844427 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: Python main interpreter initialized at 0x55e019a4f9c0 May 19 21:46:29.844427 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: python threads support enabled May 19 21:46:29.844427 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: your server socket listen backlog is limited to 100 connections May 19 21:46:29.844427 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: your mercy for graceful operations on workers is 90 seconds May 19 21:46:29.844812 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: mapped 403077 bytes (393 KB) for 2 cores May 19 21:46:29.844886 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: *** Operational MODE: preforking *** May 19 21:46:29.845131 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: *** uWSGI is running in multiple interpreter mode *** May 19 21:46:29.845131 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: spawned uWSGI master process (pid: 80196) May 19 21:46:29.845338 dv-cinder-842633-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 21:46:29.846221 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: spawned uWSGI worker 1 (pid: 80206, cores: 1) May 19 21:46:29.846221 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: spawned uWSGI worker 2 (pid: 80207, cores: 1) May 19 21:46:29.846351 dv-cinder-842633-1-iscsi devstack@keystone.service[80196]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 21:46:35.545525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:46:35.545525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes May 19 21:46:35.546319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:46:35.546319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes May 19 21:46:35.593604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:46:35.593604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes May 19 21:46:35.594178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:46:35.594178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes May 19 21:46:35.596085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 21:46:35.596276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 21:46:35.596466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] command line args: [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 21:46:35.596759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 21:46:35.596938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ================================================================================ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 21:46:35.597189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.597430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_dir = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.597613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.597792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_source = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.597969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] debug = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.598166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.599565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_config_append = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_dir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.600774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.601021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.601198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.601373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.601615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.601858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.602086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.602316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.602544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.602772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.603017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.603264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.603533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.604931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.605114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] publish_errors = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.605347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.605575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.605751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.605993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.606224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.606456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] standard_threads = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.606684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.606923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.607097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.607344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_journal = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.607591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_json = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.607879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_stderr = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.608055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_syslog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.608285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.608541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.608806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.609066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.609309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.609557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.609794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.610042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.610279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.610525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.610770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.611018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.611249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.611493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.611726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.611969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.612217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.612451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.612726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.612982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.613224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.613457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.613707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.613966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.614217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.614462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.614691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.614918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.615146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.615378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.615625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.615870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.616098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.616325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.616569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.616841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.617075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.617322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.617549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.617783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.618015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.618242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.618469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.618696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.619162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.619600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.619833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.620067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.620295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.620529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.620793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.621040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.621270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.621504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.621738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.621969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.622201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.622435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.622661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.622906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.623137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.623373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.623604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.623861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.624095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.624324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.624556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.624817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.625083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.625319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.625565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.625810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.626043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.626275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.626507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.626755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.626990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.627224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.627467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.627701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.627937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.628169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.628401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.628652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.628918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.629164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.629405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.629639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.629879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.630113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.630345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.630578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.630804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.631031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.631258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.631489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.631716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.631952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.632186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.632432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.632688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.633062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.633324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.633566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.633796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.external = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.634024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.634319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.634552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.634784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.password = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.635040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.token = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.635474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.635738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.635989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.636222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.636471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.636750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.636980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.637214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.637460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.637709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.637957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.638192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.638435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.638666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.638928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.639188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.639426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.639678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.639962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.640207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.640466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.640732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.640988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.641220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.641453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.641694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.641935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.642163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.642405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.642636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.642886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.643118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.643369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.643603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.643846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.644137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.645979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.646322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.646478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.646685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.646840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.647072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.647314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.647557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.647809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.648053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.648320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.648554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.648813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.649047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.649279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.649505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.649736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.649974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.650212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.650990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.651227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.651369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 21:46:35.651498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 21:46:35.651641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.651768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] command line args: [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 21:46:35.651891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.652023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 21:46:35.652148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.652272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ================================================================================ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 21:46:35.652403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.652528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.652686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.652829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_dir = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.652954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.653088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.653221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.653347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_source = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.653470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.653593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] debug = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.653593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.653865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.653992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.654122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.654247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.654379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.654504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.654648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.654776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.654900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.655024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.655147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.655270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_config_append = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.655401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.655526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.655526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_dir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.655761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.655885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.656008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.656139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.656280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.656405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.656528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.656669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.656802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.657013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.657144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.657268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.657399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.657523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.657656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.657787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.657787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.658052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.658178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.658300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.658435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.658561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.658684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.658807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.658931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.659070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.659204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.659340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.659466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.659589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.659589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.659839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.659965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.659965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.660221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.660347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.660474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] publish_errors = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.660610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.660803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.660803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.661047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.661178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.661302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.661444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.661580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.661705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.661828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.661952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] standard_threads = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.662076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.662207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.662338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.662463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.662586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.662709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.662833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.662964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_journal = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.662964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_json = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.663253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.663380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_stderr = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.663504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.663629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_syslog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.663761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.663887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:46:35.664023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.664936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.665951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.666088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.666816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.666954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.667985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.668931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.669836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.670970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.671904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.672942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.673889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.674983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.675915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.676896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.677873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.678943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.679741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.680956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.681087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.681087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.681347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.681347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.681619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.682320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.682497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.682632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.682772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.682772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.683902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.684912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.685972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.686931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.687882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.688971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.689898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.690978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.external = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.password = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.token = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.692199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.693312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.694405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.695568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.696675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.696675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.696675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.696675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.696675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.697324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.697324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.697324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.698208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.698348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.698482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.698698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.698936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.699186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.699440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.699674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.699906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.700137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.700381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.700662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.700915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.701169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.701415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.701685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.701923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.702162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.702393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.702629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.702859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.703094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.703334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.703594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.703818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.704048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.704294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.704541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.704800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.705048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.705293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.705528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.705765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.705993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.706229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.706464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.706707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.706944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.707191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.707440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.707688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.707918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.708149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.708395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.708664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.708903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.709139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.709375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.709606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.709847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.710106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.710336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.710581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.710828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.711073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.711308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.711537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.711771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.712004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.712238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.712470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.712742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.713063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.713364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.713614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.713863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.714094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.714340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.714577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.714814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.715049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.715284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.715536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.715782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.716019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.716217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.716301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.716451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.716543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.716706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.716793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.716967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.717202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.717439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.717686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.717927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.718159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.718394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.718645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.718894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.719144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.719381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.719629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.719880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.720127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.720374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.720634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.720871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.721106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.721356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.721588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.721837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.722085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.722334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.722580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.722828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.723073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.723319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.723566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.723797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.724043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.724290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.724536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.724814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.725221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.725300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.725464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.725713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.725962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.726210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.726458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.726692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.726946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.727200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.727436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.727668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.727899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.728130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.728380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.729169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.729689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.729927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.730197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.730431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.730666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.730902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.731152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.731389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.731646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.731894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.732133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.732376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.732631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.732888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.733125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.733364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.733597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.733854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.734102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.734333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.734576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.734878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.735142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.735388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.735635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.735880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.736123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.736359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.736596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.736853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.737095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.737335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.737572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.737815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.738050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.738297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.738533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.738776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.739006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.739246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.739478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:46:35.739648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 21:46:35.754806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.755035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.765026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.765261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.765490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.775411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.794531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.794766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.808606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.832026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.859884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.860119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.869485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.869724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.925770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.925997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.926943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.927069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.927292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.950389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.950644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.950829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.950920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.951926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.952146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.962851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.963310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.975721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.975955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.976181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.976405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.976687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.976869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.977087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.977441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.977674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.977901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.978123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.978341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.979073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.979297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:46:35.997369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:35.997579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:46:35.997911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:35.998110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:46:35.998374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:35.998626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:35.998875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:35.999123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.000185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.000398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:46:36.000760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.000962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:46:36.001227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.001484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.001738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.001991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.010306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.010573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.010845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.011124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.014748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.015041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.015352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.015593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.019518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.022883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.023157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.023434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.023829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.023914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.024107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.024363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.024646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.024932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.025199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.025493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.025758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.027391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.027673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.027954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.028350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.028652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.028923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.029196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.029474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.029742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.030016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.030283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.052932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.054531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.054807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.058290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.060122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.060354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.060640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.060937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.061213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.067092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.067374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.067656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.068701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.068983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.073610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.073885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.074160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.074438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.074710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.074978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.075063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.075239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.075327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.075500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.079916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.080294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.080779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.081224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.081647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.082036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.082425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.082828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.095247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.095517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.100019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.103113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.103389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.103697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.104108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.108270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.109321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.111909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.112201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.113033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.116886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.117183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.117280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.117467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.117747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.118019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.118290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.121327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.125070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.125365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.125645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.125923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.126195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.126471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.138259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.138629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.138892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.139152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.145631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.146007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.146275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.146549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.147112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.148857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.149131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.152806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.153080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.153373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.153649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.153921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.155172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.156930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.157205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.160238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.160533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.160839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.161137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.161413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.167489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.167758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.168032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.168307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.168582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.168875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.169155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.174852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.175132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.175419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.175697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.175973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.176245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.176518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.188240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.191510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.194819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.195192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.196413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.198251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.198501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.198670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.198780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.202021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.203622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.205299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.205585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.205865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.206321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.207918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.211153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.211436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.211714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.211989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.214006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.215615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.218871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.219160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.219444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.219720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.221975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.222308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.222628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.229076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.229613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.229897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.230994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.231268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.231550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.231822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.232093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.232380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.232676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.232941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.233207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.233484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.233757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.234011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.234275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.234528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.237940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:46:36.238219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.238494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.238773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.239045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.239316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.239591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.239855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.240128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.240400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.240710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.240972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.241240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.241503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:46:36.273988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e019a4f9c0 pid: 80206 (default app) May 19 21:46:36.277855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166286-0a11-4b88-a44c-220601cfb33f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:36.278146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166286-0a11-4b88-a44c-220601cfb33f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:36.278423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166286-0a11-4b88-a44c-220601cfb33f None None] PATH_INFO: `/v3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:36.281687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e019a4f9c0 pid: 80207 (default app) May 19 21:46:36.282828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Thu May 19 21:46:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:44.418336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:44.418897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:44.418897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:44.423264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:46:44.428644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:44.429065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:44.429192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:44.508262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: INFO dbcounter [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] Registered counter for database keystone May 19 21:46:44.529892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG oslo_db.sqlalchemy.engines [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80206) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 21:46:44.536060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writer thread running {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 21:46:44.711154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80206) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 21:46:44.711332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80206) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 21:46:44.768065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:44.769696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:44.900503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 21:46:45.044003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:45.112552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: INFO dbcounter [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] Registered counter for database keystone May 19 21:46:45.132732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG oslo_db.sqlalchemy.engines [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80207) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 21:46:45.138941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writer thread running {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 21:46:45.257196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846ecca-b623-4f86-b335-47e1d6c249cd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:45.262241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:45.262902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:45.263190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:45.263469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:45.271553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:45.324405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING keystone.common.rbac_enforcer.enforcer [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 21:46:45.324788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddf3e9-8620-4ab6-b217-6905cad713af admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:45.335441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Thu May 19 21:46:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:46:45.342321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:45.360168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cb90a-a863-41f9-a135-aa913245cd2b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:45.364577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:45.365761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:45.366053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:45.366334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] PATH_INFO: `/v3/projects/906e152e20d14899a1c568a7ddf7bb54` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:45.367869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] RBAC: Authorizing `identity:get_project(project_id=906e152e20d14899a1c568a7ddf7bb54)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:45.528891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 21:46:45.529491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a968e-b928-473b-8770-951e6dcb178b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:45.533979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Thu May 19 21:46:45 2022] GET /identity/v3/projects/906e152e20d14899a1c568a7ddf7bb54 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:47.758565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee997c8-2ba9-4cca-9f4c-5defc24485a5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:47.759077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee997c8-2ba9-4cca-9f4c-5defc24485a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:47.759182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee997c8-2ba9-4cca-9f4c-5defc24485a5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:47.763198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:47.768570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:47.769005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:47.769166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:47.788531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:47.790107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:47.798685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 21:46:47.821291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee997c8-2ba9-4cca-9f4c-5defc24485a5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:47.833149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee997c8-2ba9-4cca-9f4c-5defc24485a5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:47.837303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:47.838421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:47.838702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:47.838967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:47.846490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:47.848240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16318c5-27ca-4336-a95f-0cd4ed2faf4a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:47.857701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Thu May 19 21:46:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:46:47.862977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:47.871924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714feb2d-cfa8-4de5-9c44-4e3c9d4d7190 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:47.876294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:47.877190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:47.877480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:47.877759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:47.879143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:47.881035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e80623-637e-4ae1-8773-400be86439ef admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:47.882594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Thu May 19 21:46:47 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:50.192299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6192-491f-4a9e-bd67-3fed0dd4f797 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.192832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af3ba-4df9-4353-b49e-529bd6d347f9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.193201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6192-491f-4a9e-bd67-3fed0dd4f797 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.193292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af3ba-4df9-4353-b49e-529bd6d347f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.193369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f6192-491f-4a9e-bd67-3fed0dd4f797 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.193449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af3ba-4df9-4353-b49e-529bd6d347f9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.194390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:50.194494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:46:50.199448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.199721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.199820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.200008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.200099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.200292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.217327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:50.218505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:50.240712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80207) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 21:46:50.241129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80207) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 21:46:50.287496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:50.289115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:50.296939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 7/12] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:46:50.319006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.342513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.391540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 6/13] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:50.413926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.423637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51ad2d-ecbe-44f3-a490-4b044ce5880e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.458294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.458938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.459226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.459494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.517526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7167507-282d-48ef-8236-b6b5bf36dc06 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.519787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.527338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:50.528860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:50.551463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.555349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.556434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.556881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.557088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.568507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:46:50.580087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:46:50.581289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:46:50.582296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:46:50.587093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.628201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 8/15] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:46:50.633657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.686454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d142-584e-4bcb-8025-eaee56760f2f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.720222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.720945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.721242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.721521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.722270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:50.722571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:50.723998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:50.729635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c54b-875c-48da-9473-7f55d2abff2a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.735991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 8/16] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:50.741518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.753109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.766291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.767081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.767589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.767704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.769651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.778303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:50.779800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:46:50.801096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:46:50.802133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:46:50.802913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:46:50.802913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:46:50.802913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:46:50.802913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:46:50.802913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:46:50.808850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.842292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc480749-a0a2-461b-b4b3-091feeb9e818 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.876027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.876935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.877223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.877504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.879099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:50.880561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:50.882404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 9/18] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:50.887818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:50.912503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74310d5e-6a98-48d3-8088-ec4db697a5a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:50.955749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:50.956420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:50.956736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:50.957022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:50.957761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:50.958068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:50.959491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:50.977167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 10/19] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:50.991137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:50.999195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90cd81-260a-4574-9283-ee07fc650ef9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:51.036671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:51.037500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:51.037788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:51.038068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:51.039357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:51.041497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf7928-74a5-4b48-bd81-f4db359c64d7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:51.043388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 10/20] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:51.049387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:51.085619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:51.159435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede389-f1d1-4f52-bad3-a259e1aa3fa1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:51.197526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:51.198161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:51.198446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:51.198722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:51.199555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.domain.id=default, user.id=d32395c520864a849d965edd5ff1c3ef) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:51.199866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:51.201356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbbe4b-0b04-4ba7-9876-c6b64aa5aba0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:51.208275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 11/21] 10.222.0.61 () {56 vars in 1322 bytes} [Thu May 19 21:46:51 2022] GET /identity/v3/role_assignments?user.id=d32395c520864a849d965edd5ff1c3ef&scope.domain.id=default&role.id=6a9f236c49c24283afb390181f99e500 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:53.844535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca66bf-2ef8-4479-b6e0-c7d4afe75fa3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:53.845587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca66bf-2ef8-4479-b6e0-c7d4afe75fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:53.846072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca66bf-2ef8-4479-b6e0-c7d4afe75fa3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:53.847928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 11/22] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:53.853364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69966013-6588-4ded-ba80-9c6534da2a74 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:53.854173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69966013-6588-4ded-ba80-9c6534da2a74 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:53.854923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69966013-6588-4ded-ba80-9c6534da2a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:53.874477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-69966013-6588-4ded-ba80-9c6534da2a74 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:53.875677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-69966013-6588-4ded-ba80-9c6534da2a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:53.967451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 12/23] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:53.981540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a838-045a-44de-b356-e5d36657b827 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:53.982132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a838-045a-44de-b356-e5d36657b827 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:53.982608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a838-045a-44de-b356-e5d36657b827 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:53.984442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 12/24] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:53.989381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:53.989678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:53.989962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.008498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62a838-045a-44de-b356-e5d36657b827 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:54.013960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:54.023405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:54.056875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:54.147138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 13/25] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:54.182265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:54.214753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62a838-045a-44de-b356-e5d36657b827 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:54.273593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:54.274653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.276343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.276820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.446369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c0e47-153f-4443-b686-47703fcfb69b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:54.461466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:54.463730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-34dad65f-7aa9-4be5-beee-4c9bb3deb555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:46:54.485315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:46:54.486145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:46:54.486929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:46:54.486929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:46:54.486929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:46:54.486929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:46:54.488663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 13/26] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:46:54.490139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397f812-bb86-47b3-9ff9-152f37033f71 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.490581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397f812-bb86-47b3-9ff9-152f37033f71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.490993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397f812-bb86-47b3-9ff9-152f37033f71 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.492543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 14/27] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:54.494331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9c33-6433-480e-be20-70b83a2187a9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.494823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9c33-6433-480e-be20-70b83a2187a9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.495305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9c33-6433-480e-be20-70b83a2187a9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.496717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 15/28] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:54.497919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb9c33-6433-480e-be20-70b83a2187a9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:54.499630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:54.500494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.500974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.501402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.502335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:54.503753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6154f-642b-4e8e-93ef-08d62eb96ded None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:54.517000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:54.612475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb9c33-6433-480e-be20-70b83a2187a9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:54.652148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:54.652471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 14/29] 10.222.0.61 () {60 vars in 1131 bytes} [Thu May 19 21:46:54 2022] POST /identity/v3/roles => generated 219 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:46:54.653357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.653890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.654417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.654631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.654724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.654990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.655846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:54.656359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:54.658115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f1bc9-b6c5-4a4e-aa26-6c52d1ba4790 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:54.691198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:54.693472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:54.694540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 16/30] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:54.699079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d53a1-ef0c-4892-9454-7522be55fae9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.699729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d53a1-ef0c-4892-9454-7522be55fae9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.700202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d53a1-ef0c-4892-9454-7522be55fae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.727492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:54.738065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d28d53a1-ef0c-4892-9454-7522be55fae9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:54.741859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d28d53a1-ef0c-4892-9454-7522be55fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:54.817343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 15/31] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:54.825806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:54.845888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 17/32] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:54.848557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd990e4f-fe59-4859-80ff-9b5bb50d16f1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:54.849298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd990e4f-fe59-4859-80ff-9b5bb50d16f1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:54.850922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd990e4f-fe59-4859-80ff-9b5bb50d16f1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:54.858265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 18/33] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:54.858265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd990e4f-fe59-4859-80ff-9b5bb50d16f1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:54.865524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:54.973712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e5f85-fef7-4723-9d04-8c7efa087607 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:54.999249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd990e4f-fe59-4859-80ff-9b5bb50d16f1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.013353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.014035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.014343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.014616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.015373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.017023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0870-3565-421d-8cf7-5fd0901f7086 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.043856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.044326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.045040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.045334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.045642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.046510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.047983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.062382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 16/34] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:46:54 2022] POST /identity/v3/projects => generated 301 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:46:55.062382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.062382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.062382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.083630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:55.085042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:55.170281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 17/35] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:55.171148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.186484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 19/36] 10.222.0.61 () {60 vars in 1131 bytes} [Thu May 19 21:46:54 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:46:55.203720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.285937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.326744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6a02e-1a84-4a96-ab07-9b5464a245ae None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.364659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71f5a-a25c-42d8-bd0f-9871b215c5e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.372095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.378458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.379292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.379584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.379873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.396080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:46:55.406358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:46:55.407297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:46:55.408036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 18/37] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:46:55.413794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.418879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.420063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.420554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.421085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.422706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.424801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.426887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 20/38] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:55.433612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.540918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952eac3-bf02-4714-9579-573f3590649f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.557068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5a9d3-c325-4002-bf12-d7b6897b76e2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.575426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.576095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.576472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.576796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.577553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:55.577860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.579267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88844-79fb-4ff0-9ccf-bc376cea535b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.587222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.595860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 19/39] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:55.595860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0f47-eea8-46e6-98f9-4a91ef83c8d4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.595860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0f47-eea8-46e6-98f9-4a91ef83c8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.595860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0f47-eea8-46e6-98f9-4a91ef83c8d4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.595860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 20/40] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:55.596526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.596697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0f47-eea8-46e6-98f9-4a91ef83c8d4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.597231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.597533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.597822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.598551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.600061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622449-bdad-43b1-868c-fbeb4f1c4615 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.629101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 21/41] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:46:55 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:46:55.630411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.630703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.630985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.654578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:55.656427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:55.663038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.724571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0f47-eea8-46e6-98f9-4a91ef83c8d4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.743619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 22/42] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:55.766344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.766526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.767433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.767724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.768006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.769410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.770931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.772668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 21/43] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:46:55 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:55.779700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.889600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386191-eb4f-4429-be91-d316ba186323 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.893075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdaaa-e17b-4afc-b2fc-8ec4bef97658 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:55.905451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.918648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:55.927970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.928762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.929131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.929426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.930186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.931828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74804102-4c21-4981-b2d1-d23ab74b78ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.935042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:55.935858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:55.936158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:55.936442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:55.937849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:55.939496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:55.941578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 22/44] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:46:55.947388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:55.950911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 23/45] 10.222.0.61 () {60 vars in 1131 bytes} [Thu May 19 21:46:55 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:46:56.068423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8061e-b911-41a3-9457-b22b2a6e8301 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:56.106795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:56.107717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:56.108004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:56.108287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] PATH_INFO: `/v3/domains/default/users/d32395c520864a849d965edd5ff1c3ef/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:56.110447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d32395c520864a849d965edd5ff1c3ef, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:56.111974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59119c85-a357-483b-a07e-ede46552a688 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:56.127448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 23/46] 10.222.0.61 () {58 vars in 1361 bytes} [Thu May 19 21:46:55 2022] PUT /identity/v3/domains/default/users/d32395c520864a849d965edd5ff1c3ef/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:46:57.893529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82081830-bcf0-434e-b7c0-e25da445a0b4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:57.894080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82081830-bcf0-434e-b7c0-e25da445a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:57.894080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82081830-bcf0-434e-b7c0-e25da445a0b4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:57.895476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 24/47] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:57.901546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:57.901950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:57.902184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:57.918695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:57.919884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:57.961961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:57.987272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 24/48] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:58.009977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82081830-bcf0-434e-b7c0-e25da445a0b4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.113807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82081830-bcf0-434e-b7c0-e25da445a0b4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:58.146839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:58.147595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.147879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.148152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.149425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:58.151544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942eac6-331c-404b-9cc6-88bdf4b0b39b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:58.153310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 25/49] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:58.159078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.259616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3114-330f-4149-aa6a-8f5af5d134fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:58.293258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:58.293920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.294213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.294494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.295215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:58.296724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3596da-9a50-464b-b7ce-b4cbde1d4b45 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:58.321317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 25/50] 10.222.0.61 () {60 vars in 1140 bytes} [Thu May 19 21:46:58 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:46:58.346082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b70eb1-3ab5-472e-a72f-a79b25e77170 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.346381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b70eb1-3ab5-472e-a72f-a79b25e77170 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.346669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b70eb1-3ab5-472e-a72f-a79b25e77170 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.348090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 26/51] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:46:58.355275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.355568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.355854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.373543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:58.374781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:58.402283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:58.449156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 26/52] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:46:58.473325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b70eb1-3ab5-472e-a72f-a79b25e77170 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.521081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:58.582739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b70eb1-3ab5-472e-a72f-a79b25e77170 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:58.616744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:58.617539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.617830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.618110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.628231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:46:58.636442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:46:58.637425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:46:58.638168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 27/53] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:46:58.643388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.743740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d16b7-d734-4a40-9da1-df652fd49d34 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:58.777670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:58.778332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.778615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.778892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.779638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:58.779945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:58.781484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:58.789426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 27/54] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:58.794902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.896090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09326550-ed10-4cb1-8df3-bc35dfe05c41 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:58.897682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:58.931606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:58.932479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:58.932827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:58.933093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:58.934298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:58.935752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:58.937416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 28/55] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:46:58 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:58.942780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:58.968248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:59.052117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73397-19ef-4f82-bdb2-1240b6ee1f6c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:59.085418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:59.086213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:59.086500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:59.086780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:59.088209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:59.089704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de58189-cb32-4ad6-b5fe-c71ce210ab6a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:59.095978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 28/56] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:46:59.097106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:59.197337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec89858-50cf-4cfe-ab86-7cd7981b0cf3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:59.231820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:59.232472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:59.232866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:59.233155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:59.234002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.domain.id=default, user.id=d32395c520864a849d965edd5ff1c3ef) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:46:59.234325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:59.235783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ad649-40cf-4ef1-8cce-25614e439a4e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:59.244524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 29/57] 10.222.0.61 () {56 vars in 1322 bytes} [Thu May 19 21:46:59 2022] GET /identity/v3/role_assignments?user.id=d32395c520864a849d965edd5ff1c3ef&scope.domain.id=default&role.id=6a9f236c49c24283afb390181f99e500 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:00.559975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f778-227d-4f88-9b5a-072c60e7efb1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:00.560515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f778-227d-4f88-9b5a-072c60e7efb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:00.560515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47f778-227d-4f88-9b5a-072c60e7efb1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:00.561931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 29/58] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:00.567238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:00.567603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:00.567817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:00.584963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:00.586152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:00.612275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:00.652392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 30/59] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:00.676155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47f778-227d-4f88-9b5a-072c60e7efb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:00.766409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:00.777479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47f778-227d-4f88-9b5a-072c60e7efb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:00.810434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:00.811161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:00.811451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:00.811732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:00.812950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:00.814410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa4a00-d21e-440d-8c77-8d821e99493f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:00.816018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 30/60] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:00.821361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:00.919648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2893fa-4024-45cf-aa10-11ec47e0b253 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:00.952596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:00.953280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:00.953573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:00.953859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:00.954579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:00.956027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7042d82-2806-4d2d-8692-756132e632b6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:00.980949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 31/61] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:00 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:04.252544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3059-5a0c-413d-a66c-119c80a83b4c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.253117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3059-5a0c-413d-a66c-119c80a83b4c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.253117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3059-5a0c-413d-a66c-119c80a83b4c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.254779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 31/62] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:04.260135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.261024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.261507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.283719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241fbd-8749-4ad1-902d-0f0fb485bc24 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.284299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241fbd-8749-4ad1-902d-0f0fb485bc24 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.284802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c241fbd-8749-4ad1-902d-0f0fb485bc24 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.285452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:04.286623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 32/63] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:04.286734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:04.292795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.293348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.293849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.310191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:04.320566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:04.322042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:04.408062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 32/64] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:04.444425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:04.445857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 33/65] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:04.470920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:04.604934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa78a62-0f0e-429f-bddc-d0239caed633 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:04.629424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:04.642369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:04.643309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.643695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.644072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.644826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483be74f-b450-4b6a-908e-11d8465ef273 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:04.658013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bd9894ee-26e1-4843-9f7f-6124d846988f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:04.667919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:04.669023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:04.669890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:04.670794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 33/66] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:04.672663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244496-cc7b-4600-89b4-5d038bc99808 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.673030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244496-cc7b-4600-89b4-5d038bc99808 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.673632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244496-cc7b-4600-89b4-5d038bc99808 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.675328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 34/67] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:04.677437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb244496-cc7b-4600-89b4-5d038bc99808 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:04.693905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:04.694985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.695181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:04.695556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.696116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.707595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-39d6c23d-750e-4ecb-84fc-1448a2f63bb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:04.716448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:04.717564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:04.718341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:04.718895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 34/68] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:04.724531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.724935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.725291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.743351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:04.745267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:04.816550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb244496-cc7b-4600-89b4-5d038bc99808 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:04.835649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 35/69] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:04.835649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:04.872045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:04.872664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:04.872961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:04.873247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:04.874058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:04.874315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:04.875725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:04.886379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 35/70] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:04.887276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:04.944937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:04.970952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13818f-2d87-45e1-bb89-4f77e3aa2539 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:04.974846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.007248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.007924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.008024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b18e57-6425-4d40-8915-49ad81b8a91b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.008213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.008494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.009865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:05.010180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.011613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.022138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 36/71] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.022969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.061197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.062012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.062297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.062767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.073695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fbf921bd-f222-4294-a45f-2471c4cff943 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:05.081103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:05.081962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 36/72] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:05.088656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313c2f-4735-4ea8-8bcf-976a9bb2e058 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.089339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313c2f-4735-4ea8-8bcf-976a9bb2e058 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.089339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313c2f-4735-4ea8-8bcf-976a9bb2e058 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.089339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 37/73] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:05.089339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313c2f-4735-4ea8-8bcf-976a9bb2e058 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.154600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ff5b-50b1-4a36-937f-8f2fd3abac5f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.190336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.191224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.191519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.191796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.201084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313c2f-4735-4ea8-8bcf-976a9bb2e058 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.203148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.207892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.209893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.216310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 37/74] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.216310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.238112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.244721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.245596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.245883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.246182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.247364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.248836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e8f0c-58b7-4765-9bad-f2aaab40ac1c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.250653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 38/75] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.251740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.252026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.252304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.268881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:05.270062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:05.325007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b8507-a3a7-4ef9-af33-6cdc21227316 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.340331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 39/76] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:05.341075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.361372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.362054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.362346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.362701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.363464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:05.363779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.365217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.377396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 38/77] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.377396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.431660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.442300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06986fa8-7ee4-4ff9-aa15-59f11e5f07c6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.472140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.475166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c96dbf-9450-4b48-a56e-4a5708b4faba None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.480959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.481612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.481907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.482196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.508202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.509417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.509711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.509998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.551767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.553298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5076f0-bfe6-41b2-85d9-d14cff6a7246 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.577951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.579436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.636410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.654583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 40/78] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:05.655912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a0e65-aa9c-4be6-9f49-93db0a6cb7b3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.656211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a0e65-aa9c-4be6-9f49-93db0a6cb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.656499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a0e65-aa9c-4be6-9f49-93db0a6cb7b3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.657844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 41/79] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:05.659811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a0e65-aa9c-4be6-9f49-93db0a6cb7b3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.683013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 39/80] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:05.684117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.692781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.765846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a0e65-aa9c-4be6-9f49-93db0a6cb7b3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.781705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e6376-992d-40aa-b382-c6e1119e2675 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.800006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.800826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.801110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.801387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.812072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:05.815480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.816338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.816638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.816926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.818113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.819573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28089-6a38-4f46-aab5-9048b7f5cdfb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:05.819668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:05.820563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:05.821939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 42/81] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:05.827436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 40/82] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.827505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.827568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.827568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.827568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.841291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:05.842477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:05.864418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.908792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 41/83] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:05.909612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:05.919259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:05.931734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050539d3-f0a8-4550-9d26-4c085f495ab6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:05.966091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:05.966880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:05.967171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:05.967455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:05.968700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:05.970159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:05.976418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 43/84] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:05.976418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.011469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b027fc7-bc2e-4b27-ba91-73540c9b523f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.045969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.046835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.047121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.047402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.048591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.050112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.051975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 42/85] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.052580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.078485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5b1ea-c4df-4bcf-be9b-fd39bf632367 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.078642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.112290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.112958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.113243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.113522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.139670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.151003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb8b61-f909-44a9-94b6-c653b76f46f6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.181741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.183208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.185816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.186487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.186776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.187060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.187815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:06.188126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.189536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.197865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 43/86] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.198726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.281746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 44/87] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:06.282517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.288307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.302087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70c2f-be4e-4e9b-997d-1eb741f4ae2f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.336560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.337690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.337984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.338279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.339467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.340508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.340970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.342756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 44/88] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.343684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.381954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7167c61-5f79-4f0e-b492-5f5999ad656c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.416288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.417047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.417401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.417745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.418645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:06.419034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.420512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.426573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 45/89] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.427655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.445332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca622a2-f291-4819-a736-8ad6628d0e3a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.486211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.486901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.487193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.487481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.488374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=d32395c520864a849d965edd5ff1c3ef) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:06.488706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.490262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.498099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 45/90] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/role_assignments?user.id=d32395c520864a849d965edd5ff1c3ef&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.498852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.538994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.566002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc18da-fa0b-4237-8f54-d69a818946f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.600691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.615403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.616425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.616760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.617046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.619107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.620809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.622614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 46/91] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.623383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.660962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e36d2-2495-4663-932b-81966c6601ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.702353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.703647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.704184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.704706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.706348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.708091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.710459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 46/92] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:06.711505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.773009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861c844-6efe-4fdb-9721-8a4112a471dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.809933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:06.822580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.823122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c64cd4-2477-4b45-a991-c1cf3b843a3c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:06.823260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.823541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.823825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.824557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:06.827305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:06.857473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 47/93] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:06 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:06.858300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:06.878122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:06.878797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:06.879084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:06.879366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:06.916852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:07.009581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:07.011965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:07.018906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f246b-47bd-4960-99ed-fb1605963681 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:07.115009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:07.115588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:07.115874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:07.116153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:07.118194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:07.118522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:07.120029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:07.126033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 48/94] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:07.149067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:07.153411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 47/95] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:07.160841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:07.287859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9955f7-eb79-4319-a670-d5f1581b6c16 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:07.300052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:07.328067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:07.328985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:07.329276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:07.329560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] PATH_INFO: `/v3/projects/77d9c8af865441b682bfcd42fe959ea4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:07.330850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] RBAC: Authorizing `identity:get_project(project_id=77d9c8af865441b682bfcd42fe959ea4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:07.332464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d49d-859a-4aab-9bdf-05053513571f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:07.334285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 49/96] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:07 2022] GET /identity/v3/projects/77d9c8af865441b682bfcd42fe959ea4 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:07.340351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:07.465169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56712ffb-4051-44b8-ad0f-44e79ba207b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:07.508226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:07.509223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:07.509439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:07.509698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:07.510553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=77d9c8af865441b682bfcd42fe959ea4, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:07.510872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:07.512349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5c52-6534-4aec-8314-713a4bc8ac25 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:07.518344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 48/97] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:07 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=77d9c8af865441b682bfcd42fe959ea4&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:09.126465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006b53-db6e-4232-8f6e-f363fbab7ed9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.127431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006b53-db6e-4232-8f6e-f363fbab7ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.127913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006b53-db6e-4232-8f6e-f363fbab7ed9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.129792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 50/98] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:09.136074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114df2-fd4b-4385-a7ff-e30c247d7d52 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.136935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114df2-fd4b-4385-a7ff-e30c247d7d52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.137445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114df2-fd4b-4385-a7ff-e30c247d7d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.155694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5d114df2-fd4b-4385-a7ff-e30c247d7d52 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:09.157228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5d114df2-fd4b-4385-a7ff-e30c247d7d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:09.215912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:09.249777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 49/99] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:09.256562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2c16-caf9-420c-a3e3-95e97cbc3d26 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.257094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2c16-caf9-420c-a3e3-95e97cbc3d26 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.257576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2c16-caf9-420c-a3e3-95e97cbc3d26 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.259225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 51/100] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:09.264364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.264923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.265427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.286536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:09.287727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:09.314398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe2c16-caf9-420c-a3e3-95e97cbc3d26 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:09.375256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 50/101] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:09.410196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:09.524897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe2c16-caf9-420c-a3e3-95e97cbc3d26 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:09.565399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:09.566450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.566883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.567483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.579519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:09.588855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:09.589842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:09.590599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:09.592832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 52/102] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:09.602163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:09.607172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17d01c-5014-4cdb-81a7-f0cd4c20fc59 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:09.640984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:09.672218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:09.672916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:09.673749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.674038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.674340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.690882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:09.704496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:09.705412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:09.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 51/103] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:09.712257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:09.807484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c840a-d2e9-451f-a994-cd7a55e0de09 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:09.893579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:09.894179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:09.894471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:09.894754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:09.895513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:09.895823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:09.897231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:09.912398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 53/104] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:09.920918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:09.960410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8092-e5cc-404a-9d6d-295b6330e832 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.027527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.028218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.028506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.028819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.029585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:10.030217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.037348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.054775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 52/105] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:10.060995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.129642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:10.138468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7dc8-34de-45f2-bc4e-3a6b6bfe2a53 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.148098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:10.231106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220026d-7144-49b9-a027-5b99cb9feea4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.242361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.243777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.244063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.244341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.248899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.250614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.252607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 54/106] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:09 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:10.257830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.302254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.303146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.303430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.303705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.305374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.306880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.313011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 53/107] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:10 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:10.314381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.473944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000499-9484-47dc-9ad8-9dce4a0585f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.477280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:10.488720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549ce3-3af9-43ce-bd66-6ccf834bac4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.523560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.525900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.526741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.530330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.532249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.534081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19676a-ef58-49cb-9c78-9c6ac2e3e082 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.536445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 54/108] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:10 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:10.539421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94c49-6959-473e-b63f-2fbbc841d17d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.540270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94c49-6959-473e-b63f-2fbbc841d17d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.542391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94c49-6959-473e-b63f-2fbbc841d17d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.544983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 55/109] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:10.546779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae94c49-6959-473e-b63f-2fbbc841d17d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.575140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.576490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.577041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.577526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.580330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.582165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f381d-0bac-4aa9-a6ce-6ffe76d63d72 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.584258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 55/110] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:10 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:10.590648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.592067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.594354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.628126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:10.631052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:10.678863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:10.753394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 56/111] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:10.756659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.767971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae94c49-6959-473e-b63f-2fbbc841d17d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.810486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.811398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.811690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.811975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/d32395c520864a849d965edd5ff1c3ef/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.814094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, user_id=d32395c520864a849d965edd5ff1c3ef, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.816261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238512a-c2e0-4d6d-aa8e-358f6f0f1cbb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.832566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 56/112] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:10 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/d32395c520864a849d965edd5ff1c3ef/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:10.833981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c5ed-9196-4c83-bca9-5066c4149618 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.834270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c5ed-9196-4c83-bca9-5066c4149618 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.834553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3c5ed-9196-4c83-bca9-5066c4149618 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.836194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 57/113] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:10.841923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3c5ed-9196-4c83-bca9-5066c4149618 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:10.891080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86dc83-e12d-413a-bab1-e01e0b8a8ab8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:10.893706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:10.933557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:10.934862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.935334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.935795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.938233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:10.940044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6864124-a1a7-4dc8-8d11-80a788cd0b6f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:10.960443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 57/114] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:10 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:10.963176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e03d5-98b3-40a6-93ce-7ae9042642bb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.963771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e03d5-98b3-40a6-93ce-7ae9042642bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.964257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e03d5-98b3-40a6-93ce-7ae9042642bb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.966217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 58/115] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:10.967836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c60-750d-45fb-b432-b3aa5352646b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:10.968368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c60-750d-45fb-b432-b3aa5352646b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:10.968961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04c60-750d-45fb-b432-b3aa5352646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:10.989850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ddf04c60-750d-45fb-b432-b3aa5352646b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:10.991688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ddf04c60-750d-45fb-b432-b3aa5352646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:10.999850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3c5ed-9196-4c83-bca9-5066c4149618 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.028460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.069455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.070394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.070756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.071144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.085044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:11.086684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 59/116] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:11.088554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081296c-1d3e-4314-95bb-778d9b477598 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.088854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081296c-1d3e-4314-95bb-778d9b477598 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.089198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081296c-1d3e-4314-95bb-778d9b477598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8a9f095c-0870-422c-9a09-d4ad76854cf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:11.094959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:11.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:11.096630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 58/117] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:11.102873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5e4a-ed97-4b5c-8602-60a278533f0e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.102873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5e4a-ed97-4b5c-8602-60a278533f0e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.103110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5e4a-ed97-4b5c-8602-60a278533f0e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.106092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 59/118] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:11.107065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe5e4a-ed97-4b5c-8602-60a278533f0e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.120068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b081296c-1d3e-4314-95bb-778d9b477598 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:11.122407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b081296c-1d3e-4314-95bb-778d9b477598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:11.224702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 60/119] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:11.226487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.226785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.227071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.235614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.248810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:11.248974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe5e4a-ed97-4b5c-8602-60a278533f0e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.250005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:11.292733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.293415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.293706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.294127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.294894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:11.295356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:11.299227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:11.310818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 60/120] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:11.312677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.333834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 61/121] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:11.335078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.355079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.442118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afe1c-1c5e-41bf-9ab3-1480dee5d3b1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.446055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d3421-2d9f-4202-a0f6-8c2b0a588dcf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.482506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.483265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.483934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.484127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.484419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.484739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.485016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.485496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.486988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:11.488789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:11.490627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 62/122] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:11.491978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.500759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:11.510028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:11.510950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:11.512100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 61/123] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:11.512758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.537845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.597064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.608042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3527a-bf72-4fbd-aec6-4b3af25ffe3f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.646071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.647493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.647979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.648481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.650150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:11.650771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507943d9-29c9-48f6-aead-1489a262f3d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.652208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:11.654208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 63/124] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:11.655288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.696970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.698307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.698597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.698877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.713831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:11.724587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:11.725475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 62/125] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:11.726663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.773412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04772c-b3d2-48e4-8460-f3fa629a1a15 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.811542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.812610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.813120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.813629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:11.816195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:11.828735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.847175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 64/126] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:47:11 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:11.848553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.853212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:11.865757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d1446-81ed-4ab0-98f0-41f2296354f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:11.906946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:11.907956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:11.908415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:11.908901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:11.909942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:11.910512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:11.912199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:11.922423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 63/127] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:11.923720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:11.972754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2aa731-3a55-49ed-87c7-c63fc4a15d69 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.011910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.013124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.013489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.013842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] PATH_INFO: `/v3/projects/77d9c8af865441b682bfcd42fe959ea4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.020817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=77d9c8af865441b682bfcd42fe959ea4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.022463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.036682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 65/128] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/projects/77d9c8af865441b682bfcd42fe959ea4 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:12.037480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.038611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e699c2-524b-4078-bb59-426d05065262 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.078851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.080025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.080554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.081065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.082077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:12.082619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.084309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.086327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:12.097343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 64/129] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:12.098596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.161374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:12.210046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97802-900c-471a-b6ae-f86a9b3723bf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.228189 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c62ac-7e89-4950-a600-601c079bf5e2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.248636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.250180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.250657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.251194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] PATH_INFO: `/v3/projects/77d9c8af865441b682bfcd42fe959ea4/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.253374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d9c8af865441b682bfcd42fe959ea4, user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.255195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.267373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.268247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.268524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.268843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.270066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.271530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.272480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 65/130] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:12 2022] PUT /identity/v3/projects/77d9c8af865441b682bfcd42fe959ea4/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:12.273443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 66/131] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:12 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:12.273894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.279777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.358447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:12.431334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5794-6401-4370-81d7-3ac3778fd3c0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.519552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada691-6c17-4cff-8d10-2a394f0d6ecb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.526922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.527829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.528116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.528399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] PATH_INFO: `/v3/users/f78c276582e949b6999cc0f060907177` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.561380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] RBAC: Authorizing `identity:get_user(user_id=f78c276582e949b6999cc0f060907177)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.562899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.564836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 66/132] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:12 2022] GET /identity/v3/users/f78c276582e949b6999cc0f060907177 => generated 289 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:12.572609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.598274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.599423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.599862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.600287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.601696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.603453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.605669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 67/133] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:12 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:12.611386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:12.639283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:12.752028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6796a-17c2-4b86-9fe3-4e917feda346 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:12.786015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:12.804234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:12.804931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:12.805231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:12.805636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:12.909638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:12.911251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:12.965865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4ac64-67a0-4f30-8124-0c76396230b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:13.016685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 67/134] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:13.023754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:13.046737 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:13.048549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.049033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.049471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.049587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:13.054482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02, role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:13.057080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:13.090954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 68/135] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:12 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:13.180036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecdc43-9444-4429-b92d-cfb66e6940b1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:13.232759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:13.233657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.233947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.234234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.235464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:13.236984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ce475-2ed4-4f4c-b23c-86cf867701e6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:13.238698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 68/136] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:13 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:13.245003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:13.347993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:13.397650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd2ed-4784-405a-9a6a-9dfb53d8e1dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:13.451023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:13.451680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.451982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.452269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.453237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=f78c276582e949b6999cc0f060907177) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:13.453565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:13.455018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe26b67-7e76-4a47-a2bd-a7d6b3929f12 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:13.460893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 69/137] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:13 2022] GET /identity/v3/role_assignments?user.id=f78c276582e949b6999cc0f060907177&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:13.931812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf28-992b-4de9-bb2f-12f3ea295204 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.932332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf28-992b-4de9-bb2f-12f3ea295204 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.932414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440bf28-992b-4de9-bb2f-12f3ea295204 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.933942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 69/138] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:13.939155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.939633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.939933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.967227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:13.968437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:13.994507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02606ec-41d2-4b39-b470-b786f7d1b2f4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:13.995158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02606ec-41d2-4b39-b470-b786f7d1b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:13.995654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02606ec-41d2-4b39-b470-b786f7d1b2f4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:13.997277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 70/139] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:14.014109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.014723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.015221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.057702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:14.059376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:14.114177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 70/140] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:14.148899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.198660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 71/141] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:14.254929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:14.260816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.291758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:14.315078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc132-e430-442d-9bcd-4c530e7c7c00 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:14.385739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:14.386713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.387014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.387298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.409768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:14.418519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:14.419432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:14.420223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 71/142] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:14.429589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.497096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613d9f-4625-47bc-83e0-204d7d96956d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:14.589748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:14.590671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.590959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.591236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.613809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:14.621085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb051c-2fbb-4be5-a0b9-7891d382e4a8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:14.630178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:14.631077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:14.631813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 72/143] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:14.647743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.670247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:14.686527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:14.687264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.687561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.687850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.688641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:14.688960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:14.690357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:14.704457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 72/144] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:14.714308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.774019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:14.823682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4c895-46ba-4457-af80-47acf45f9820 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:14.867614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6b77-fdfa-47b2-a1e8-da4a174dd6fa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:14.894705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:14.895383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.895675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.895957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.896738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:14.897051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:14.898452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:14.903664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:14.904666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:14.904966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:14.905248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:14.906497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:14.907963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:14.909649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 73/145] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:14.913699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 73/146] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:14.920004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.922273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:14.978985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.032037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40727-b0a3-4a6e-8b36-51f259f742ea None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.093185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d811176-bd8a-41dd-b9a2-e4005a7209e8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.093716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.094225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.094520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.094806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] PATH_INFO: `/v3/users/d32395c520864a849d965edd5ff1c3ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.104642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d32395c520864a849d965edd5ff1c3ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.104642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.105167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 74/147] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/users/d32395c520864a849d965edd5ff1c3ef => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:15.113549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:15.164006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.165118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.165605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.166087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.181588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.288180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.290016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca931ffc-4b97-4bf7-b9ab-c96faff6bc53 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.334559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422895a-8375-49a0-9214-b0c48fd660d1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.371200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.399014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.400662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.400961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.401244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.521033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 74/148] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:15.523335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26aa6-c64c-4b16-8530-3b5dd375f40e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.523916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26aa6-c64c-4b16-8530-3b5dd375f40e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.525210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26aa6-c64c-4b16-8530-3b5dd375f40e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.529255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 75/149] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:15.535165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d26aa6-c64c-4b16-8530-3b5dd375f40e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:15.538007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.542953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01137b12-60cf-4631-af18-656dc327bb5a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.591563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.653728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 75/150] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:15.655346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.655987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.656456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.676745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:15.678807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:15.681206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d26aa6-c64c-4b16-8530-3b5dd375f40e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.696767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.733633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.734822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.735300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.735747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.737586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.739613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6056a44-a482-4902-88ee-6f7e589ad2cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.742164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 76/151] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:15 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:15.745794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f898-1fcb-4ee1-9558-b1add9f8c88a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.746342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f898-1fcb-4ee1-9558-b1add9f8c88a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.746767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f898-1fcb-4ee1-9558-b1add9f8c88a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.748722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 77/152] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:15.749723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9f898-1fcb-4ee1-9558-b1add9f8c88a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:15.776659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 76/153] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:15.784759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:15.894116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9f898-1fcb-4ee1-9558-b1add9f8c88a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.895848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:15.908745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f033-5a3d-4a94-bac5-453da6144447 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:15.934874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.935745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.936027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.936304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.937547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.939019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166e9c6-2ef9-4f9e-bbed-4f003db084a7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.940758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 78/154] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:15 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:15.942078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.942546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.942964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.948758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:15.949768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:15.950246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:15.950743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:15.951866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:15.952430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:15.954219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:15.960550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:15.961522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 77/155] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:15 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:15.962984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:15.963359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:15.981926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.058410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 79/156] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:16.059177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.095920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa360c-af75-4747-958a-0a9df18c03cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.144597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.145848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.146344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.146862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.148873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:16.151598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:16.154633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 78/157] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:16.155620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.182739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.187151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b9ec6-e9d1-4f04-bcf2-a4ce99b172cc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.237447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.238565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.238951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.239328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.241134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=d32395c520864a849d965edd5ff1c3ef) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:16.241573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:16.243826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:16.253136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 80/158] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/role_assignments?user.id=d32395c520864a849d965edd5ff1c3ef&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:16.254321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.268214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.334647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951339d-c01e-4e26-a965-d9d05a2f671c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.381944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d342f0-45e8-4ba5-8e7c-84631064e7a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.422037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.422797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.423179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.423553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.424567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:16.426789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:16.441422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.442925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.442925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.443261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.461635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 81/159] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:16 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:16.478680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7b5e418c-8f7c-48e0-9b64-8b4a87164783 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:16.487617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:16.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:16.489752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:16.492188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 79/160] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:16.498507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.513542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8f1c1-dd87-4f2c-b1ac-f31bc8bc4ac6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.514086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8f1c1-dd87-4f2c-b1ac-f31bc8bc4ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.514574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8f1c1-dd87-4f2c-b1ac-f31bc8bc4ac6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.516141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 80/161] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:16.521801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.522378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.522867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.545200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.564913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:16.566162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:16.653750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749208d-ef86-4c75-9f95-4dd6492ceb1d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.666633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 81/162] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:16.691596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.715902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.716750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.717393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.717791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.718642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:16.718970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:16.720585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:16.732696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 82/163] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:16.738628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.786722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.823935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c62af-6589-485f-8b81-17352a0fa0f3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.843025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:16.875563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd849-a48b-41b0-aa39-668fe0f39858 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:16.876143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.876999 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.877285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.877564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.891789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7dc1ac69-927c-4093-9425-024499789fb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:16.900705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:16.901585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:16.902349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 82/164] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:16.903241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affb56-953d-41b3-8d8b-fbddc969449c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.903798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affb56-953d-41b3-8d8b-fbddc969449c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.903798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affb56-953d-41b3-8d8b-fbddc969449c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.906424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 83/165] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:16.911093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46affb56-953d-41b3-8d8b-fbddc969449c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:16.924380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:16.924380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.924380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.924380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.926027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:16.927118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634174e-e66d-4f71-a966-a9a425bbb2a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:16.929060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 83/166] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:16.931065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:16.931338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:16.931621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:16.950180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:16.951351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:17.034528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46affb56-953d-41b3-8d8b-fbddc969449c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.045678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 84/167] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:17.046440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.081543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:17.089693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288225-21b0-4865-ac93-09653309315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.090377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288225-21b0-4865-ac93-09653309315d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.090672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288225-21b0-4865-ac93-09653309315d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.091135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288225-21b0-4865-ac93-09653309315d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.092464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288225-21b0-4865-ac93-09653309315d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:17.092931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288225-21b0-4865-ac93-09653309315d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:17.094325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44288225-21b0-4865-ac93-09653309315d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:17.105586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 84/168] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:17.106432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288225-21b0-4865-ac93-09653309315d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.154753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:17.179820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db64ad1-76e4-47cc-b665-4cb7afc5a3a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.227062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.228933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.229468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.229746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.267207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288225-21b0-4865-ac93-09653309315d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.309739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.310547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.310829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.311103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.338052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:17.346222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:17.347143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 85/169] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:17.347931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.356489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:17.358067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:17.445640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:17.466858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 85/170] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:17.467789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.534316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:17.577139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd569f5-4503-46e8-bf30-fe6129b676cd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.619579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.620597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.622139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.622790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.625096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:17.626391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:17.628665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:17.642121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af3410-139d-43ef-95c2-024980974ae6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.652243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 86/171] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:17.653386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.712281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.715174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.716294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.717492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] PATH_INFO: `/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.719824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd78c6ca86e4f6e93ecc5bd9b90cb02)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:17.722238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:17.723918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 86/172] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/users/7cd78c6ca86e4f6e93ecc5bd9b90cb02 => generated 275 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:17.724727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.734853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:17.830335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89308a-0228-481d-9711-8cb02ee0389a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.885047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:17.885920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:17.886203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:17.886476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] PATH_INFO: `/v3/projects/77d9c8af865441b682bfcd42fe959ea4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:17.887688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] RBAC: Authorizing `identity:get_project(project_id=77d9c8af865441b682bfcd42fe959ea4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:17.889352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:17.891056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 87/173] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/projects/77d9c8af865441b682bfcd42fe959ea4 => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:17.892023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:17.930647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1127bc-9bcc-4c03-8916-17b690944355 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:17.976967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.008019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.009470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.009760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.010038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] PATH_INFO: `/v3/users/f78c276582e949b6999cc0f060907177` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.011427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709f55-3169-43d9-894b-2c43868287b2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.017419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] RBAC: Authorizing `identity:get_user(user_id=f78c276582e949b6999cc0f060907177)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.019030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.021211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 87/174] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/users/f78c276582e949b6999cc0f060907177 => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:18.022055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.023073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:18.049735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.050514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.050860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.051249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.130282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.132412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.235968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 88/175] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:18.237222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:18.257026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.330714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b686a-b8c4-48a3-9ee1-c87be3f68163 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.395814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae5e9-8941-4549-9cff-81d9ca5f3a7b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.404924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.405598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.405889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.406176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.407033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=77d9c8af865441b682bfcd42fe959ea4, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:18.407354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.408971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.416144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 88/176] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:18 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=77d9c8af865441b682bfcd42fe959ea4&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 608 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:18.417393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:18.469066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.471517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.472253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.473462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.476545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.478578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.482113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 89/177] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:18 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:18.488860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:18.506445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.577577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26c7b1-6ca3-4140-906b-e805e60e6d37 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.619200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.620382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.621054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.621519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.623838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.625712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.628062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 89/178] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:18 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:18.638928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:18.654608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.714554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfe92c-f821-45ec-ba5d-8433444b469c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.792007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.793182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.793471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.793752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/f78c276582e949b6999cc0f060907177/roles/4f1be953ef494f47b9acaf1429ebd5ca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.801484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, user_id=f78c276582e949b6999cc0f060907177, role_id=4f1be953ef494f47b9acaf1429ebd5ca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.802996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a180d-34ac-4d34-a025-5413d837d3a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.839316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 90/179] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:18 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/users/f78c276582e949b6999cc0f060907177/roles/4f1be953ef494f47b9acaf1429ebd5ca => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:18.841220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef657fa5-27c1-4189-9c31-a3d690e2375b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:18.879063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:18.899396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:18.900496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:18.900953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:18.901381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:18.902482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:18.903068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:18.904847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5cd49-dec5-4b3b-b728-9dac8e41f89d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:18.915393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 90/180] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:18 2022] GET /identity/v3/role_assignments?user.id=7cd78c6ca86e4f6e93ecc5bd9b90cb02&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:19.349650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ec75-a556-4b94-9526-15f340e251e1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.350185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ec75-a556-4b94-9526-15f340e251e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.350185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ec75-a556-4b94-9526-15f340e251e1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.351692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 91/181] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:19.358161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd248e0-291e-42f6-aa8b-8894a77f666c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.358726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd248e0-291e-42f6-aa8b-8894a77f666c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.359208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd248e0-291e-42f6-aa8b-8894a77f666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.377606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4cd248e0-291e-42f6-aa8b-8894a77f666c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:19.379143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4cd248e0-291e-42f6-aa8b-8894a77f666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:19.467509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 91/182] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:19.495887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70ec75-a556-4b94-9526-15f340e251e1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:19.625462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70ec75-a556-4b94-9526-15f340e251e1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:19.638893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ff617-977d-4582-b4aa-28695089ffe6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.639407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ff617-977d-4582-b4aa-28695089ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.639687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ff617-977d-4582-b4aa-28695089ffe6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.641103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 92/183] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:19.646173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd892-1ce7-4792-bd65-17761a8ffc2b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.646700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd892-1ce7-4792-bd65-17761a8ffc2b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.647193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd892-1ce7-4792-bd65-17761a8ffc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.671382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:19.672596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.672980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a04bd892-1ce7-4792-bd65-17761a8ffc2b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:19.673127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.673419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.674715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a04bd892-1ce7-4792-bd65-17761a8ffc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:19.682709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:19.687263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:19.697901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:19.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:19.699653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:19.702060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 92/184] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:19.709514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:19.765988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 93/185] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:19.767434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f3f7-6ba7-4ba9-bf92-ab7ae515d2ff None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.767731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f3f7-6ba7-4ba9-bf92-ab7ae515d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.768014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1f3f7-6ba7-4ba9-bf92-ab7ae515d2ff None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.769388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 94/186] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:19.775129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.775424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.775712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.794632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:19.795960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:19.832103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2498-bed9-4e45-a9a7-f359e1dbf9d1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:19.845572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:19.867458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:19.868845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:19.869294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:19.869583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:19.870323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:19.870632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:19.872039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:19.875304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 95/187] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:19.876459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:19.885271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 93/188] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:19.890629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:19.961250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:19.991772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785661c-1d4b-4e84-ba85-be75334e2584 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.005656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd0a65-86f1-4e16-af5c-81102b71240d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.037389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.038408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.038851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.039122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.040652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.041326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.042197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.042476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.042733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.042842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.044576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 96/189] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:20.045689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.060789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.062397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.068321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 94/190] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:19 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:20.068321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.102124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.161927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c13ce-2868-4b03-8397-f47cb6b7190b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.171388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b955ee-8f61-4c23-b090-79e82183f5e9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.190178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.202816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.203638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.203924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.204200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.210307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.211153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.211968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.212245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.213006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.214464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.215610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:20.223787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:20.224700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 97/191] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:20.226288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.240500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 95/192] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:20 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:20.253743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.370210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa77b2-836e-46e2-aa71-e38df8491385 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.385699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d730f51-e862-4a96-80c6-94437d645f87 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.389713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.420469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.422128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.422716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.423148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.423918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:20.424328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.426175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.427252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.427973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.428380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.428801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.437672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 96/193] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:20.443323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.496518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.514604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.516699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.575164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423184-1647-4234-b4ad-e4e859ded8e8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.614374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.615597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.616175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.616456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.618331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.620150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.621883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 97/194] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:20.627201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.645176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 98/195] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:20.651846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.656664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.747336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fbf76-0838-4a47-9308-18ca7b65243a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.749677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.771919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106eed-6354-4279-8534-ce71ed67c416 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:20.783861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.784518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.784823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.785354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.819175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:20.821907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:20.822630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:20.823769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:20.825902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.827360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.829471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 99/196] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:20.835202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:20.868382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:20.870209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:20.919965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:20.946458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889bd4c-7865-4952-b8fd-e87b0a422a50 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:21.003613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:21.004202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.004483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.004788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.005633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=d9f9ad276cb449caba4c261eb272b823) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:21.005947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:21.007385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:21.014328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 100/197] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/role_assignments?user.id=d9f9ad276cb449caba4c261eb272b823&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:21.062849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 98/198] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:21.068743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:21.200177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177e06-248e-4364-a625-4f143493b522 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:21.240101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:21.241030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.241317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.241596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.243575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:21.245085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718e15f-0c97-4e0f-8092-9496c4a7c2d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:21.246918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 101/199] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:21 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:21.253125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:21.262209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:21.379061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68a8e1-b9c6-4f65-94fc-39afbda2f480 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:21.416799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:21.417459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.417743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.418020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.418868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=d9f9ad276cb449caba4c261eb272b823) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:21.419177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:21.420667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5f1db-6616-47c9-85f5-4348ea5df6c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:21.426534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 99/200] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:21 2022] GET /identity/v3/role_assignments?user.id=d9f9ad276cb449caba4c261eb272b823&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:21.440142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aaea3-6a67-4f59-be94-37a2f520838c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.440470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aaea3-6a67-4f59-be94-37a2f520838c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.440730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aaea3-6a67-4f59-be94-37a2f520838c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.442183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 102/201] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:21.448321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.448687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.448921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.494560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:21.495835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:21.594552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:21.600056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 100/202] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:21.637722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aaea3-6a67-4f59-be94-37a2f520838c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:21.641813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:21.767839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aaea3-6a67-4f59-be94-37a2f520838c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:21.803043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:21.803850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:21.804139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:21.804415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:21.823167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:21.831259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:21.832249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:21.833067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 103/203] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:21.841786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:22.002415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9ade5-6481-41ec-a76b-404c2f1d7430 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:22.057709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:22.058387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:22.058674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:22.058955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:22.059720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:22.060027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:22.061507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:22.070423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 101/204] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:22.076415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:22.166121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:22.216883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60026-97da-4f7e-8fc4-0b640de67555 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:22.273424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:22.274933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:22.275446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:22.275948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] PATH_INFO: `/v3/users/f78c276582e949b6999cc0f060907177` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:22.277544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] RBAC: Authorizing `identity:get_user(user_id=f78c276582e949b6999cc0f060907177)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:22.279098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:22.281062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 104/205] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:22 2022] GET /identity/v3/users/f78c276582e949b6999cc0f060907177 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:22.286932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:22.394003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:22.450922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747969d0-c41c-481c-93ec-bd5c2d876b2f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:22.508361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:22.509658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:22.509953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:22.510241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:22.588993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:22.590857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:22.653811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:22.700894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 102/206] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:22.706075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:22.871155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b99b98-69f3-4bcf-b79b-15f1c9eaf9fc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:22.905770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:22.917083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:22.918265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:22.918898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:22.919413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:22.925377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:22.927100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e070547-39b2-417f-9c0b-c4bcd6d36620 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:22.929831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 105/207] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:22 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:22.936107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:23.082561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9b574-deb4-4401-8cd0-e3859953010a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:23.120104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:23.120803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.121090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.121426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.122220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c, user.id=f78c276582e949b6999cc0f060907177) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:23.122535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:23.124079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201b3c-216d-4d13-99c9-2146bfdb2bf6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:23.131005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 103/208] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:22 2022] GET /identity/v3/role_assignments?user.id=f78c276582e949b6999cc0f060907177&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:23.237517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b64c5-fdfb-49ef-ac50-86334c41dd0a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.238120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b64c5-fdfb-49ef-ac50-86334c41dd0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.238552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b64c5-fdfb-49ef-ac50-86334c41dd0a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.240107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 106/209] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:23.246176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.246464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.246914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.274437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:23.276086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:23.297390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:23.373734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 104/210] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:23.417991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b64c5-fdfb-49ef-ac50-86334c41dd0a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:23.546743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b64c5-fdfb-49ef-ac50-86334c41dd0a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:23.585390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:23.586279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.586642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.587224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.597704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:23.608470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:23.609511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:23.610738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 107/211] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:23.615604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:23.728254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f689a-6065-471e-8433-9f35c002ef5f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:23.770759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:23.771473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.771748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.772024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.772854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:23.773159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:23.775034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9178e1-a5c9-439b-b7a7-e00339cb8418 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:23.784565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 105/212] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:23.790157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:23.860992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:23.897179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577154b-8905-448d-9e90-f491aca0010e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:23.932906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:23.933787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.934073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.934358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] PATH_INFO: `/v3/users/fea7533051e840a881aa7036875c80a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.941598 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de4d9-8795-45a9-b9ba-b45acb64129a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.942113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de4d9-8795-45a9-b9ba-b45acb64129a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.942558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de4d9-8795-45a9-b9ba-b45acb64129a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.944337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 106/213] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:23.949603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad8d8-e658-4e85-8f2d-168e2ff7e112 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.949890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad8d8-e658-4e85-8f2d-168e2ff7e112 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.950168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad8d8-e658-4e85-8f2d-168e2ff7e112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.951142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=fea7533051e840a881aa7036875c80a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:23.952602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713b7a9-ecdb-495a-84b4-73abbf196ff7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:23.954469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 108/214] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:23 2022] GET /identity/v3/users/fea7533051e840a881aa7036875c80a0 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:23.955586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285286f6-7041-4443-a7b9-a7ce671c36bf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:23.955874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285286f6-7041-4443-a7b9-a7ce671c36bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:23.956152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285286f6-7041-4443-a7b9-a7ce671c36bf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:23.957624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 109/215] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:23.960251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285286f6-7041-4443-a7b9-a7ce671c36bf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:23.972515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-660ad8d8-e658-4e85-8f2d-168e2ff7e112 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:23.973826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-660ad8d8-e658-4e85-8f2d-168e2ff7e112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:23.986513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.060649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 107/216] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:24.060649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.060649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.060649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.071071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285286f6-7041-4443-a7b9-a7ce671c36bf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.080202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:24.082272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:24.093579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.114739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.115439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.115702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.115981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.160010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 108/217] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:24.166149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.194601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.197281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.255799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.282980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27be49-dd9c-4b3c-8b9b-954a551d63f7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.303903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 110/218] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:24.304778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.319695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.320489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.321337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.321617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.323125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.331889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-930c2b18-86e3-402a-88d9-1047e7120e80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:24.340263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:24.341185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:24.342003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 109/219] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:24.343081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f9f0-efc2-4a5f-bd9d-b55d95649d58 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.343174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f9f0-efc2-4a5f-bd9d-b55d95649d58 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.343384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f9f0-efc2-4a5f-bd9d-b55d95649d58 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.344801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 110/220] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:24.346372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72f9f0-efc2-4a5f-bd9d-b55d95649d58 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.405526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88b1b-ad11-4c53-9a6e-704bda421a1c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.439304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.440099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.440385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.440696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.441867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.443319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.445124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 111/221] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:24.446333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.451549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72f9f0-efc2-4a5f-bd9d-b55d95649d58 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.484991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.485844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.486127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.486402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.487575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.489073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15121458-3f10-419d-b243-e88890faa2b4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.490749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 111/222] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:24.491877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.492164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.492438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.498407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.509352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:24.510474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:24.535095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.555291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338fd21-6f7d-4959-93ca-8adaaf493cfc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.576228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 112/223] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:24.577074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.592531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.593235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.593516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.593792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.594543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:24.594877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.596236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.605356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 112/224] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:24.606146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.683208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942684-7ee0-4020-bdbb-1914f908659b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.702885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6039-c50f-4f64-a0e8-fd1583b183c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.719003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.719685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.719969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.720251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.720994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.722463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.729416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.739233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.740009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.740294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.740569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.741436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=fea7533051e840a881aa7036875c80a0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:24.741762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.756716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 113/225] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:24 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:24.756944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.757039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.757109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 113/226] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/role_assignments?user.id=fea7533051e840a881aa7036875c80a0&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:24.757109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.767589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:24.869156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3a279-d360-4af2-80c8-b280bf52019f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.889753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015b92a-b95d-403f-bbc4-50a3ad54a0de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:24.912783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.914192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.915175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.916552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.918114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:24.920349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:24.923122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 114/227] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:24.934017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:24.937333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:24.938173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:24.938485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:24.938759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:24.951998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:24.960303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:24.962354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:24.963330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 114/228] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:24.968914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:25.042773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:25.043755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:25.074978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58668ad-d0bb-4a6a-99cb-f29cc3a05228 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.116248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fddc-2139-4399-97e9-0d571698b545 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.118703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.119850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.120370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.120913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.122338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.123973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.126625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 115/229] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:25.133477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:25.167566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.168348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.168941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.169041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.169780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:25.170093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.172556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.183450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 115/230] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:25.189761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:25.287885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2ffc6-2dca-4d35-9c1e-bb3b521e8014 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.322143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db932-37ea-4e91-9fcd-3258248eb478 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.336057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.337546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.339152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.341754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/d9f9ad276cb449caba4c261eb272b823/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.342099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:25.345373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, user_id=d9f9ad276cb449caba4c261eb272b823, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.346859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.373366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.374308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.374629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.374954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.375052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 116/231] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:25 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/d9f9ad276cb449caba4c261eb272b823/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:25.376491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.378198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.380104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 116/232] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:25 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:25.386259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:25.409033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:25.549623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41411657-42a1-4d2f-bb27-294f10b91b76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.594905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.595797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.596081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.596361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.601518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.603857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b3ddf-27fd-4a65-b10a-278daa25ff33 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.606017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 117/233] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:25 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:25.612012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:25.743380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d69851-9fb7-4018-8f2b-05168e5f6172 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:25.779484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:25.780374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:25.780679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:25.780962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/d9f9ad276cb449caba4c261eb272b823/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:25.783170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, user_id=d9f9ad276cb449caba4c261eb272b823, role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:25.784644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2915f74-d455-45e6-8403-6e1bfa6a0310 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:25.800223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 117/234] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:25 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/d9f9ad276cb449caba4c261eb272b823/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:27.468408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513974-5cd0-4d74-91bf-28aa654712ef None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:27.468948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513974-5cd0-4d74-91bf-28aa654712ef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:27.469023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513974-5cd0-4d74-91bf-28aa654712ef None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:27.470363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 118/235] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:27.476509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf820c-f19e-4304-8096-edddfaf653a5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:27.476509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf820c-f19e-4304-8096-edddfaf653a5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:27.476509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf820c-f19e-4304-8096-edddfaf653a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:27.505842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-08cf820c-f19e-4304-8096-edddfaf653a5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:27.507442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-08cf820c-f19e-4304-8096-edddfaf653a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:27.553580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:27.614297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 118/236] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:27.653427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01513974-5cd0-4d74-91bf-28aa654712ef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:27.762880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:27.797835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdc25-e2ce-43f9-a8de-2387d81d1f25 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:27.798251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdc25-e2ce-43f9-a8de-2387d81d1f25 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:27.798536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdc25-e2ce-43f9-a8de-2387d81d1f25 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:27.799954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 119/237] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:27.806658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01513974-5cd0-4d74-91bf-28aa654712ef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:27.806973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:27.807265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:27.807547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:27.838365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:27.839658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:27.852905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:27.853716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:27.854001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:27.854531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:27.879492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:27.898389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:27.899482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:27.901050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 119/238] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:27.910414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:27.975449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 120/239] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:28.002665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.091110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fe428-e969-470d-bfec-2de213e9b11d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.127108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.128154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.128446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.128749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.129658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:28.130072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.131622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5354-44d3-415e-a092-6a2088b74fe3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.135007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.137535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.142970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041febe-d78e-4938-b8fe-281ca5a1dd74 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.147677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 120/240] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.147677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236c5d-11e9-49f9-bb43-f41f04c88663 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.147677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236c5d-11e9-49f9-bb43-f41f04c88663 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.147677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236c5d-11e9-49f9-bb43-f41f04c88663 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.151649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 121/241] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:28.152525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae236c5d-11e9-49f9-bb43-f41f04c88663 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.189535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.190943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.191757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.192333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.210539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-34d425ce-a0a4-4345-8f61-e10f594bb7a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:28.221073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:28.222062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:28.222859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:28.227672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 121/242] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:28.230207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.230998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.231508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.252124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:28.253781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:28.292671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae236c5d-11e9-49f9-bb43-f41f04c88663 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.337016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.338051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.338482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.338667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] PATH_INFO: `/v3/users/fea7533051e840a881aa7036875c80a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.338924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 122/243] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:28.340200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.345587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] RBAC: Authorizing `identity:get_user(user_id=fea7533051e840a881aa7036875c80a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.348596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.359680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 122/244] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/users/fea7533051e840a881aa7036875c80a0 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.361686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.443207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.465821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.478559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82710a-1006-49c5-91ab-7ac8ea1450c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.484476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65fd8e-c671-4af4-9236-404c5cfd639c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.515955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.516637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.516942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.517221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.517976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:28.518281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.519690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cadd5-a8cd-4e66-a9ed-519f65d60bb1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.520668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.521523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.521818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.522088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.523257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.525477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.527173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 123/245] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.527927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.532924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 123/246] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.534406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d03b0-9674-4a8f-9979-a3c89161180f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.534700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d03b0-9674-4a8f-9979-a3c89161180f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.534980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d03b0-9674-4a8f-9979-a3c89161180f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.536268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 124/247] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:28.537365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d9b4-55fc-4e03-a326-5297d70af208 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.537649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d9b4-55fc-4e03-a326-5297d70af208 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.537921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d9b4-55fc-4e03-a326-5297d70af208 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.539279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 125/248] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:28.539889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d9b4-55fc-4e03-a326-5297d70af208 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.635000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88207dc1-247c-470b-ba40-20613b4b6c3e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.642536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d9b4-55fc-4e03-a326-5297d70af208 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.670285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.671120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.671425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.671735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.673014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.674553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.675145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.676019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.676217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 124/249] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.676326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.676554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/fea7533051e840a881aa7036875c80a0/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.678557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, user_id=fea7533051e840a881aa7036875c80a0, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.680089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e715bd-5c52-465e-8501-e0b1b71e6994 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.681248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.684078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.693449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 126/250] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:28 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/fea7533051e840a881aa7036875c80a0/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:28.694665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919817-1250-4c31-9d2e-5701240021fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.694949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919817-1250-4c31-9d2e-5701240021fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.695229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919817-1250-4c31-9d2e-5701240021fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.713867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-df919817-1250-4c31-9d2e-5701240021fd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:28.715127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-df919817-1250-4c31-9d2e-5701240021fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:28.724348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.796203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 127/251] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:28.808921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.808921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.808921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.812584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e3e89-ad16-43bd-9e0a-35546484b456 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:28.826352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:28.827648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:28.854739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:28.855926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:28.856365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:28.856661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] PATH_INFO: `/v3/users/e4827dd8b67f46ee9354e51f299ffd0f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:28.873974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] RBAC: Authorizing `identity:get_user(user_id=e4827dd8b67f46ee9354e51f299ffd0f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:28.875004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:28.877812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 125/252] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/users/e4827dd8b67f46ee9354e51f299ffd0f => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:28.878895 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.910097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 128/253] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:28.911582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:28.951090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.991897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:28.997954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7be876-89d2-41ab-8b20-d16eb13240a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.026425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c2b8-df5d-423a-9998-3080ef757e3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.042459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.043395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.043907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.044191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.044954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:29.046401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:29.066816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.067598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 126/254] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:28 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:29.068239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.068551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.069009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.069354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.092183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:29.101748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:29.103005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:29.103798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:29.107004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 129/255] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:29.108136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.220186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4278-0403-4c9c-9ab8-c7e4ab1102b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.241668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:29.244169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a6a6-2306-4348-8a43-4a0bfcbcb61b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.269438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.270471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.270961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.271494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.294468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:29.306513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.307335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.307664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.307944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.320007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:29.329854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:29.330944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:29.331902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:29.333985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 130/256] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:29.335186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.373595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:29.375063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:29.472014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edd983-4194-4e4e-babe-b28879d9695b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.509245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 127/257] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:29.510356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.516645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.517372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.517686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.517990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.518841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:29.519477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:29.520781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:29.532025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 131/258] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:29.532974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:29.533084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.619323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:29.645344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4ae1-1f6e-4241-9e74-456d1f551a92 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.695406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334f7c5-f253-42e3-9cc8-6a3d5dbd29a7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:29.710422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.711612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.712260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.712791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.714024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:29.715370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:29.717838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:29.738829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 128/259] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:29.743421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.770219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:29.771499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:29.772154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:29.774924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:29.777273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:29.779708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:29.783293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 132/260] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:29.783998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:29.899677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:29.934951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d7d8-9791-44c4-9da4-5b7daead8a94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.005855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.006654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.006934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.007210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.009131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.010592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.012796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 129/261] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:30.013687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.023482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3366d-6d0f-493a-b889-c1046b4ce35e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.059615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.060485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.060810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.061092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] PATH_INFO: `/v3/users/d9f9ad276cb449caba4c261eb272b823` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.062590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] RBAC: Authorizing `identity:get_user(user_id=d9f9ad276cb449caba4c261eb272b823)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.063119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.066200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.067855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 133/262] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:29 2022] GET /identity/v3/users/d9f9ad276cb449caba4c261eb272b823 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:30.074397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.210308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff6f6e-02fa-4761-9a6f-29767c607912 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.251730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.252394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.252746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.253025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.253915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=e4827dd8b67f46ee9354e51f299ffd0f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:30.254230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.255814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.263326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 130/263] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/role_assignments?user.id=e4827dd8b67f46ee9354e51f299ffd0f&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:30.264723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.290891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.334765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78ba88-5414-4c7e-9eef-0a73d380a81d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.425267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.425957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.426249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.426536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.446475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcf426-ccb9-4b43-9d10-d48aab1e2c27 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.511540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.512316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.512499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.512779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.550091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.581798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.583287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.637331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.638949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.659632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.752736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 134/264] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:30.754804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.828929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 131/265] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:30.840494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.914987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.934038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe971-9986-47a7-904d-0bad3710a191 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:31.013406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:31.014315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.014641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.014942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.018737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:31.018737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:31.019886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 135/266] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:31.030862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:31.034488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d4fad-7f0c-41cf-b301-9e261af689c4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:31.059054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:31.119244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:31.120459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.121001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.128639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.130565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:31.132242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:31.135628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 132/267] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:31.140987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:31.222000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b86f9-4664-4392-82ec-3ca9176c7d46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:31.304601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:31.305871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.306163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.306447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.307296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=d9f9ad276cb449caba4c261eb272b823) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:31.307611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:31.312657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae31770-2816-417f-9961-77db1f8360dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:31.325615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 136/268] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:31 2022] GET /identity/v3/role_assignments?user.id=d9f9ad276cb449caba4c261eb272b823&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:31.395186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bcb6d-6c23-49c3-a3e1-a0aa1cb51bd8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:31.436590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:31.437646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.438153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.438628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.439673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=d9f9ad276cb449caba4c261eb272b823) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:31.440184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:31.441840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8996b1-8c15-48e8-b69d-6bb1e2331484 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:31.448349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 133/269] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:31 2022] GET /identity/v3/role_assignments?user.id=d9f9ad276cb449caba4c261eb272b823&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:31.778090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d21d91-777c-41cb-aa4c-dfca8082741a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.778581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d21d91-777c-41cb-aa4c-dfca8082741a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.778656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d21d91-777c-41cb-aa4c-dfca8082741a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.780268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 137/270] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:31.787178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.787637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.787734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.806487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:31.807699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:31.845953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:31.884201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 134/271] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:31.909129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d21d91-777c-41cb-aa4c-dfca8082741a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:31.942416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.035020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d21d91-777c-41cb-aa4c-dfca8082741a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.074543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.075366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.075651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.075927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.092604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2acb1b3a-b634-4747-adce-315c3611455f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:32.101383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:32.102884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:32.103640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 138/272] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:32.109733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.240664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f6a9d-6580-4ae1-9bf5-bf07c8123300 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.282056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.283063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.283566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.284038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.285026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:32.285537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:32.287115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:32.300717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 135/273] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:32.301283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5b9d4-ab9a-4188-b6bd-ad1c2123030c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.301701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5b9d4-ab9a-4188-b6bd-ad1c2123030c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.302253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5b9d4-ab9a-4188-b6bd-ad1c2123030c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.303932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 139/274] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:32.306721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.310327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c077-927c-41a2-826a-ac732538e3a0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.310892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c077-927c-41a2-826a-ac732538e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.311414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c077-927c-41a2-826a-ac732538e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.331088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64b5c077-927c-41a2-826a-ac732538e3a0 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:32.332954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64b5c077-927c-41a2-826a-ac732538e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:32.343610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.411085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 140/275] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:32.416858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dfaf-9d77-4ca7-af66-e5754cd32382 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.416858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dfaf-9d77-4ca7-af66-e5754cd32382 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.416858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dfaf-9d77-4ca7-af66-e5754cd32382 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.416858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 141/276] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:32.420240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.420794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.421221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.424920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1107-ae8f-45fa-b3f3-961b622bf274 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.439108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:32.440359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:32.449567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.464123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.465026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.465309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.465585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] PATH_INFO: `/v3/users/fea7533051e840a881aa7036875c80a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.467029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] RBAC: Authorizing `identity:get_user(user_id=fea7533051e840a881aa7036875c80a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:32.468242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:32.470053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 136/277] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/users/fea7533051e840a881aa7036875c80a0 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:32.473430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.515955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 142/278] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:32.517494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.595096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e940-d6d4-404f-90fe-bf9a5ae850c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.597018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.631839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7dd02-305d-460b-9046-36f70c55f25e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.636639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.637608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.637891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.638164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.650907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:32.658591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:32.659782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 137/279] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:32.666779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.675048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.675724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.675999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.676285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.689275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.751853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:32.753443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:32.779218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f384bd6-4dd1-4840-9012-f33483805f5e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.818634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.819428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.819722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.820005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.822283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:32.823863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:32.830368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 138/280] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:32.831011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.861953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.870531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 143/281] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:32.870531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:32.914360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:32.955508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c9d4-ba75-413c-b7b0-db6ca1451c94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.976401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05243bae-5e7c-4fbc-ba96-82dceb139526 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:32.992960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:32.993958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:32.994251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:32.994532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:32.995287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:32.995600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:32.996994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.005166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 139/282] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.006120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.011115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.011855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.012135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.012410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.013146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.014594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879efd7d-5382-445f-b1b0-bc7c0e3d92cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.036920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 144/283] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:47:32 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:47:33.038203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c43e-ef41-4354-98a6-250067ed9f23 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.038626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c43e-ef41-4354-98a6-250067ed9f23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.038749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c43e-ef41-4354-98a6-250067ed9f23 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.040704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 145/284] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:33.041383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c43e-ef41-4354-98a6-250067ed9f23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.122840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:33.134329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b777e3b-1572-4bfa-82ec-ca055797a916 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.165371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:33.172329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c43e-ef41-4354-98a6-250067ed9f23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.173619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.174736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.175177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.175462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.176775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.178279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810966cc-d7f2-4d2d-ad72-6be4664cd347 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.180795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 140/285] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.182129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.182427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.182713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.200909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:33.202128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:33.218471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.219362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.219652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.219931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] PATH_INFO: `/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.235757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] RBAC: Authorizing `identity:get_user(user_id=a4ee7930a4bf4b31956bd0d5f6298d76)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.237285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.239316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 146/286] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.240892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.291177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 141/287] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:33.292346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.371598 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0ced0-866e-4eab-90cf-25535ed82782 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.405721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:33.418512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa34bd-c8ff-4685-b637-fab42edde9f3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.421737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.422646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.423112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.423569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.424598 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=fea7533051e840a881aa7036875c80a0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:33.424986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.426472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.433428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 147/288] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/role_assignments?user.id=fea7533051e840a881aa7036875c80a0&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.434295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.457300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.458061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.458403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.458749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.459612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, user.id=a4ee7930a4bf4b31956bd0d5f6298d76, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:33.459924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.461839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.465150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:33.467695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 142/289] 10.222.0.61 () {56 vars in 1308 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/role_assignments?user.id=a4ee7930a4bf4b31956bd0d5f6298d76&scope.system=all&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.605459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc142c6-7260-4561-a0b2-942d42e0d5f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.659162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.659888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.660168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.660440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.673880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:33.682462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:33.683463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:33.684371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 148/290] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:33.690384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.826669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b8b7c-a70f-4a02-a697-cba3c2fadb25 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:33.868344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:33.870536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:33.870827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:33.871107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:33.871873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:33.872178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:33.873571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:33.885399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 143/291] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:33.890794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:33.982047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:34.003027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97135742-f8c5-4e70-9722-60781db032cf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:34.043856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:34.044854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:34.045169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:34.045459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] PATH_INFO: `/v3/users/e4827dd8b67f46ee9354e51f299ffd0f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:34.046694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] RBAC: Authorizing `identity:get_user(user_id=e4827dd8b67f46ee9354e51f299ffd0f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:34.048321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:34.050488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 149/292] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:33 2022] GET /identity/v3/users/e4827dd8b67f46ee9354e51f299ffd0f => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:34.056907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:34.076853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:34.173556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759224-2a0c-4089-845f-7089074539f4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:34.209184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:34.210072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:34.210368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:34.210658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:34.212067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:34.213588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa4281-8532-4ad1-9477-29c4bb13f779 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:34.215466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 144/293] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:34 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:34.221298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:34.326705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035552ad-ad9a-409f-affb-cf4161614f2f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:34.368189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:34.369157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:34.369482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:34.369787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/e4827dd8b67f46ee9354e51f299ffd0f/roles/4f1be953ef494f47b9acaf1429ebd5ca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:34.372015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, user_id=e4827dd8b67f46ee9354e51f299ffd0f, role_id=4f1be953ef494f47b9acaf1429ebd5ca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:34.373631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b6299-44bb-4a34-ba7d-1c7a8c155324 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:34.387791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 150/294] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:47:34 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/users/e4827dd8b67f46ee9354e51f299ffd0f/roles/4f1be953ef494f47b9acaf1429ebd5ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:35.549496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce17b-d746-47cf-ad6f-72bca3b9993e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.550002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce17b-d746-47cf-ad6f-72bca3b9993e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.550076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce17b-d746-47cf-ad6f-72bca3b9993e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.551660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 145/295] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:35.557886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b83f-307d-43ba-af78-dbba6adcf453 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.558705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b83f-307d-43ba-af78-dbba6adcf453 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.559175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b83f-307d-43ba-af78-dbba6adcf453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.576562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6e80b83f-307d-43ba-af78-dbba6adcf453 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:35.578087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e80b83f-307d-43ba-af78-dbba6adcf453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:35.580367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:35.650320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cbe3-f445-4906-a3a9-7f17bd0060e3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.651016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cbe3-f445-4906-a3a9-7f17bd0060e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.651275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 151/296] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:35.651729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cbe3-f445-4906-a3a9-7f17bd0060e3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.654791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 146/297] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:35.659192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.659697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.660168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.675323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cbe3-f445-4906-a3a9-7f17bd0060e3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:35.678301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:35.680411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:35.766643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 152/298] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:35.785251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:35.791480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:35.808208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cbe3-f445-4906-a3a9-7f17bd0060e3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:35.859575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:35.860988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.861291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.861830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.876328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:35.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:35.887908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:35.888721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:35.889470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 147/299] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:35.894365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:35.895685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:35.903635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123f715-b4e3-4c19-ad57-41d0d310dd19 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:35.945146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:35.946180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:35.946471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:35.946746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:35.948345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:35.950225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:35.956113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 153/300] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:35.958495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.043784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24651e-d9c0-46bb-af21-5282ba0801ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.071416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493dc7e-6810-4950-995c-d8981efcb164 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.082960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.083641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.083912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.084192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.084984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:36.085303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.086736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca643c-8e75-4896-a893-cb56ef47cc45 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.099507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 148/301] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:36.099507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac147f-3e0c-4122-8d24-30cc5b696284 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.099507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac147f-3e0c-4122-8d24-30cc5b696284 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.099507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac147f-3e0c-4122-8d24-30cc5b696284 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.099507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 149/302] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:36.103161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac147f-3e0c-4122-8d24-30cc5b696284 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.112385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.114517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.115322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.115611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.115881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] PATH_INFO: `/v3/groups` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.116572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.118751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae5e6c-296a-42c6-b8a2-3f15dddcd9ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.123720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.141010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 154/303] 10.222.0.61 () {60 vars in 1134 bytes} [Thu May 19 21:47:35 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:47:36.142541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.142828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.143110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.169836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:36.171093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:36.242611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac147f-3e0c-4122-8d24-30cc5b696284 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.264645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 155/304] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:36.282158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.287571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.288456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.288833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.289900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] PATH_INFO: `/v3/users/60765e77836c4453991519c9216971f9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.311045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] RBAC: Authorizing `identity:get_user(user_id=60765e77836c4453991519c9216971f9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.312525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.314623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 150/305] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/users/60765e77836c4453991519c9216971f9 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:36.319964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.401713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.415407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961c776-a8e8-4204-8a01-232b1b8b69f9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.444920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.454237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b9dd-c8c3-4a71-b4c7-fc7c4ab74c2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.457567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.458355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.458637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.458909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.474197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:36.483505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:36.484388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:36.485200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:36.486247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 156/306] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:36.492930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.499316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.500179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.500709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.501135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.502193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, user.id=60765e77836c4453991519c9216971f9, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:36.502660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.504462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.511218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 151/307] 10.222.0.61 () {56 vars in 1308 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/role_assignments?user.id=60765e77836c4453991519c9216971f9&scope.system=all&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:36.619293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa7c-5bd1-47ea-afe1-22c13dc51985 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.659879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.660946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.661436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.661916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.662881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:36.663428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.665369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb87cc-eb2d-4a7e-9b35-d095e198b730 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.675896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 157/308] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:36.682611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.710797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dadd6-5077-45e4-9100-cd3fd119dad4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.711743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dadd6-5077-45e4-9100-cd3fd119dad4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.712233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dadd6-5077-45e4-9100-cd3fd119dad4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.713830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 158/309] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:36.721387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.721814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.722249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.740673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:36.742100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:36.793188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.814166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdabe6-8d44-46ea-8391-c4114ee14f05 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.820476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 159/310] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:36.843610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.861568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.863389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.863877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.864379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] PATH_INFO: `/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:36.866359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=a4ee7930a4bf4b31956bd0d5f6298d76)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:36.868472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:36.871561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 152/311] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:36.872798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:36.877759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:36.957171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685cd6d-1486-40bd-ac51-d8799da9c28c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:36.995785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:36.996584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:36.996919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:36.997192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.008964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:37.016087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d066-fb6f-450f-b8cb-ec0466a41dd9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:37.022636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:37.023874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:37.024988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 160/312] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:37.032119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:37.071487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:37.073621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:37.074064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:37.074519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] PATH_INFO: `/v3/system/users/a4ee7930a4bf4b31956bd0d5f6298d76/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.076293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a4ee7930a4bf4b31956bd0d5f6298d76, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:37.078024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:37.079952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:37.095186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 153/313] 10.222.0.61 () {58 vars in 1334 bytes} [Thu May 19 21:47:36 2022] PUT /identity/v3/system/users/a4ee7930a4bf4b31956bd0d5f6298d76/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:37.161982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d551-0c5f-461e-8a36-883c5aa3fc3b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:37.204582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:37.205296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:37.205578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:37.205853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.206614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:37.206974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:37.208458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:37.219080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 161/314] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:37.225690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:37.298719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:37.349829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eb63a-04b4-4fbd-86c5-cad18d8dc2b4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:37.389738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:37.391049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:37.391540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:37.392281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] PATH_INFO: `/v3/users/e4827dd8b67f46ee9354e51f299ffd0f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.393805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] RBAC: Authorizing `identity:get_user(user_id=e4827dd8b67f46ee9354e51f299ffd0f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:37.395586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:37.398092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 154/315] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:37 2022] GET /identity/v3/users/e4827dd8b67f46ee9354e51f299ffd0f => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:37.404991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:37.510464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:37.514050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7245ef-43b6-4e85-a98e-f4eacc85bc6a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:37.554889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:37.555543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:37.555825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:37.556099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.626534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:37.628028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:37.732910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:37.737515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 162/316] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:37.745351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:37.855407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a77df-b111-48da-919c-7d0c81a97498 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:37.884441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:37.894535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:37.895684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:37.896197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:37.896719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:37.898460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:37.900322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276ac9b-0f13-4dec-8f29-629193f860e6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:37.902629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 155/317] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:37 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:37.908533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:38.013245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99517e26-e586-41dc-87c9-e84ab464c214 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:38.053506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:38.054157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.054441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.055240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.056071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe, user.id=e4827dd8b67f46ee9354e51f299ffd0f) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:38.056439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:38.058303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dc70b-e60a-4071-9146-c2b88098ad1e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:38.065097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 163/318] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:47:37 2022] GET /identity/v3/role_assignments?user.id=e4827dd8b67f46ee9354e51f299ffd0f&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:38.615565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c971d-78af-4363-9b88-85c77b9f2b22 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.615964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c971d-78af-4363-9b88-85c77b9f2b22 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.616148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c971d-78af-4363-9b88-85c77b9f2b22 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.617690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 156/319] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:38.622962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276658-fba9-41ce-b05f-d6f561b8f67b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.623258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276658-fba9-41ce-b05f-d6f561b8f67b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.623530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276658-fba9-41ce-b05f-d6f561b8f67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.640771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-86276658-fba9-41ce-b05f-d6f561b8f67b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:38.641964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-86276658-fba9-41ce-b05f-d6f561b8f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:38.708123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:38.719117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 164/320] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:38.741315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c971d-78af-4363-9b88-85c77b9f2b22 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:38.843619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdda6-0625-42c9-9cae-e3b998c1e65d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.843910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdda6-0625-42c9-9cae-e3b998c1e65d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.844347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdda6-0625-42c9-9cae-e3b998c1e65d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.845879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 165/321] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:38.846926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c971d-78af-4363-9b88-85c77b9f2b22 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:38.851420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.852385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.852830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.869563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:38.870964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:38.881866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:38.883143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:38.883143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:38.883375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:38.884522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:38.885982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:38.887584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 157/322] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:38.892925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:38.941671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 166/323] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:38.962463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:38.965793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.003184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4b312-2fe4-4112-8f32-6e57faf3cf89 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.046612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.047276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.047567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.047843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] PATH_INFO: `/v3/groups` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.048531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:39.049989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:39.066279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 158/324] 10.222.0.61 () {60 vars in 1134 bytes} [Thu May 19 21:47:38 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:47:39.087384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839b88a-47ca-4acb-8931-13b4c342bae7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.114789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:39.127680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.128417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.128733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.129013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.141433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:39.149336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:39.150299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:39.156360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 167/325] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:39.157187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.274040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a8aa-5200-43af-a40d-c6f2cea753da None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.299668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:39.313665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.314393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.314764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.315054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.315813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:39.316122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:39.317527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea12ab7-601b-4cc9-be1f-3548064034bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:39.327594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 159/326] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:39.333021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.443662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd4aa-a191-467f-827f-202bca581705 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.443963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd4aa-a191-467f-827f-202bca581705 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.444217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd4aa-a191-467f-827f-202bca581705 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.446123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 160/327] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:39.448350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb456-a0f2-4b97-8826-52e8f18d04ef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.451469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.452055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.452543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.479325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:39.480889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:39.490255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.491329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.491857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.493331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] PATH_INFO: `/v3/users/60765e77836c4453991519c9216971f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.494221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] RBAC: Authorizing `identity:get_user(user_id=60765e77836c4453991519c9216971f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:39.495465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:39.497502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 168/328] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:39 2022] GET /identity/v3/users/60765e77836c4453991519c9216971f9 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:39.502791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.568724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 161/329] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:39.575887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:39.593708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.651376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a83c-124d-47c2-9ecc-241897ad6971 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.702274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.703657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.703949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.704228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] PATH_INFO: `/v3/system/users/60765e77836c4453991519c9216971f9/roles/4f1be953ef494f47b9acaf1429ebd5ca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.708595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=60765e77836c4453991519c9216971f9, role_id=4f1be953ef494f47b9acaf1429ebd5ca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:39.710069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:39.723953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 169/330] 10.222.0.61 () {58 vars in 1334 bytes} [Thu May 19 21:47:39 2022] PUT /identity/v3/system/users/60765e77836c4453991519c9216971f9/roles/4f1be953ef494f47b9acaf1429ebd5ca => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:39.730870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed1fda-bc74-4bdd-b52e-122b6ff59905 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.753776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:39.778876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.779745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.780127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.780414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.794024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:39.802426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:39.803408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:39.804290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 162/331] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:39.810969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:39.946449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637207-f635-411d-9ba3-c7791dce0038 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:39.963841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:39.986996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:39.987731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:39.988013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:39.988292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:39.989148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:39.989453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:39.990840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:40.000677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 170/332] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:40.006062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:40.136594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a322d-e057-41b3-b1f4-8c5d41ddcdd8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:40.181348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:40.182231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:40.182512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:40.182785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] PATH_INFO: `/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:40.184040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] RBAC: Authorizing `identity:get_user(user_id=a4ee7930a4bf4b31956bd0d5f6298d76)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:40.185500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6ac6e-4c80-475b-9488-362304bcd63f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:40.187136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 163/333] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:40 2022] GET /identity/v3/users/a4ee7930a4bf4b31956bd0d5f6298d76 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:40.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:40.317112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff920a-6075-4433-880f-c20f03fe1737 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:40.369378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:40.370051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:40.370341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:40.370621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:40.371467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] RBAC: Adding query filter params (role.id=c2ae6dbe1e314175bcda8fcf0e179597, user.id=a4ee7930a4bf4b31956bd0d5f6298d76, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:40.371779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:40.373455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f6f60-89cf-466d-a914-f6f6f80aeeea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:40.387349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 171/334] 10.222.0.61 () {56 vars in 1308 bytes} [Thu May 19 21:47:40 2022] GET /identity/v3/role_assignments?user.id=a4ee7930a4bf4b31956bd0d5f6298d76&scope.system=all&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:42.361679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d0ce-9d13-43ea-91c5-034d84db4634 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.362735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d0ce-9d13-43ea-91c5-034d84db4634 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.363231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d0ce-9d13-43ea-91c5-034d84db4634 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.365131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 164/335] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:42.371213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4c9-b71f-4a9b-9c3f-0eeba7ce4e9c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.372090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4c9-b71f-4a9b-9c3f-0eeba7ce4e9c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.372575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4c9-b71f-4a9b-9c3f-0eeba7ce4e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.395714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-617ea4c9-b71f-4a9b-9c3f-0eeba7ce4e9c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:42.397300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-617ea4c9-b71f-4a9b-9c3f-0eeba7ce4e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:42.434270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:42.478932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 172/336] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:42.505928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461d0ce-9d13-43ea-91c5-034d84db4634 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:42.564761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:42.646412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461d0ce-9d13-43ea-91c5-034d84db4634 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:42.669852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dd6a8-0ba4-40c4-8702-6f7ca74dba67 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.670207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dd6a8-0ba4-40c4-8702-6f7ca74dba67 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.670415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dd6a8-0ba4-40c4-8702-6f7ca74dba67 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.672071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 173/337] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:42.677751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.678038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.678320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.691532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:42.692333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.692643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.692940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.708128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:42.717254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:42.718208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:42.719089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:42.721400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 165/338] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:42.722164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:42.724651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:42.725124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:42.840307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000be9c-a4da-4a12-b589-7de882616b8c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:42.843008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:42.871244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 174/339] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:42.880009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:42.880721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.881015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.881298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.882058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:42.882367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:42.884042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:42.896805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 166/340] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:42.899072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:42.910887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.021690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3aa2-ee1f-4e53-8c37-1e6d0e51b312 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.057050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.061444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95111774-399e-41d0-96ca-6daab4b20f32 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.065790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.066792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.067308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.067797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.081176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e70be792-9402-462a-8d16-9fa9985cd164 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:43.089885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:43.090788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:43.091553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:43.094177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 167/341] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:43.095632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a675-d721-4aea-93b1-aa280dd3dce5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.096114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a675-d721-4aea-93b1-aa280dd3dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.096587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a675-d721-4aea-93b1-aa280dd3dce5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.098078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 168/342] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:43.099330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591826f-fc2e-474d-bdab-f800711bc54e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.099828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591826f-fc2e-474d-bdab-f800711bc54e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.100355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591826f-fc2e-474d-bdab-f800711bc54e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.102549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 169/343] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:43.107104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.108005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.108468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.108813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591459ae-883e-4099-bde0-38c6088d5f3a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.109081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.109285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591459ae-883e-4099-bde0-38c6088d5f3a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.109658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591459ae-883e-4099-bde0-38c6088d5f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.130821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-591459ae-883e-4099-bde0-38c6088d5f3a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:43.132356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-591459ae-883e-4099-bde0-38c6088d5f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:43.161279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.205464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:43.207413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:43.229029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 170/344] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:43.230880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.231602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.232030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.254841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:43.256131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:43.331442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.345158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 175/345] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:43.346389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.346494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 171/346] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:43.347347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.455018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.474440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd63f9-767f-4b83-b0a9-6fb06ad3e0e8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.511382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ca57f-7aa7-4253-9c58-9d6d6790e2cd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.523606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.525126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.525859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.526284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.542689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:43.559191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.560193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:43.560317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:43.561234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:43.562143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 172/347] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:43.562834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.563720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.564213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.565344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:43.565867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:43.567659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:43.579279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 176/348] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:43.580515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.646640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.705438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae11ff6-72be-4236-af48-3aa53203d3a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.705949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0e5f5-8d9a-4ca1-b9da-544f457b3e78 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.749962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.752537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.752843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.753323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.753439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.753637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.753914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.754282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.755696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:43.758366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:43.760469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 173/349] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:43.761633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.773300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:43.782289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:43.783191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:43.783989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 177/350] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:43.784687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:43.788405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.910677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fed3e1-56a6-484e-88ab-301ceb52e7e3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.915439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebedd7-fc83-47d4-925a-9fc2dd2e12d7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:43.936286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:43.955618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.956237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.956546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.956937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.957751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:43.958089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:43.959274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:43.959274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:43.959274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:43.959528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:43.959597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:43.971975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 174/351] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:43.977023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.063656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.065218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecadd3-86e4-44ac-bc8d-5d1f55bada9e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.082152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.115926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2056aaa-f612-4a87-a1f1-225a6931f6be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.168398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.169476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.169797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.170119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.177446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.178934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.187085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 175/352] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.188187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.195593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 178/353] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:44.197076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d02f-3d1d-422c-9ff7-90351291ad24 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.197486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d02f-3d1d-422c-9ff7-90351291ad24 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.197775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d02f-3d1d-422c-9ff7-90351291ad24 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.199702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 179/354] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:44.200996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402d02f-3d1d-422c-9ff7-90351291ad24 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.226855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.318915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad12d80-1181-46f4-a888-502eb86db271 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.334241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402d02f-3d1d-422c-9ff7-90351291ad24 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.370546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.371226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.371513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.371796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.372552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:44.372935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.373549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.374201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.374306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.374489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.374771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.384658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 176/355] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.385370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.388105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.458006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.459473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208c3c-9f58-4a0b-aae2-e41d60993c3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.481773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.502841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10790c0-08f8-4792-8a6b-de52f2b91135 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.545326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.545989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.546277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.546558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.548217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:44.548217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.549182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.555245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 177/356] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.555955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.594980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 180/357] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:44.597859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad00ad-ac42-46f3-90d4-0e034aa4523a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.597859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad00ad-ac42-46f3-90d4-0e034aa4523a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.597859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad00ad-ac42-46f3-90d4-0e034aa4523a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.618771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d8ad00ad-ac42-46f3-90d4-0e034aa4523a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:44.620273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8ad00ad-ac42-46f3-90d4-0e034aa4523a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:44.651988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.700135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d43e4-96a8-4294-a31d-2c21cda5b5aa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.713866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 181/358] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:44.715799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b5d21-f96b-4188-8953-157ae19579e1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.716198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b5d21-f96b-4188-8953-157ae19579e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.717038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b5d21-f96b-4188-8953-157ae19579e1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.718867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 182/359] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:44.720303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b5d21-f96b-4188-8953-157ae19579e1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.756400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.757234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.757706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.758639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.759876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.761360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.763284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 178/360] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.764202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.793194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:44.860868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b5d21-f96b-4188-8953-157ae19579e1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.898771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a3f-924b-490b-a83e-6725f340110d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.910636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.911830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.912149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.912503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] PATH_INFO: `/v3/users/60765e77836c4453991519c9216971f9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.916103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] RBAC: Authorizing `identity:get_user(user_id=60765e77836c4453991519c9216971f9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.917907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675f879-9394-4ba8-b452-d8bb4561576a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.919752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 183/361] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/users/60765e77836c4453991519c9216971f9 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.921193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.921663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.922099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.947020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.947222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:44.947890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.948175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.948450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.948558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:44.950214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.951656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.953715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 179/362] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:44.954486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.962154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.050392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 184/363] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:45.052125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.080989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2807e2-f472-4c50-ae71-fba29acf2570 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.118996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.138153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.138966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.139536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.139957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.154605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:45.166170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:45.167432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:45.168382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:45.170574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 180/364] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:45.171816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.200113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7d401-80f6-45c0-b279-5b13566efa85 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.250813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.251912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.252119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.252408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.252851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.256467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.256467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.258921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 185/365] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.263996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.315685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1784871-a45f-4981-bb43-a5e673119f65 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.368424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.369196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.369482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.369760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.370600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] RBAC: Adding query filter params (role.id=4f1be953ef494f47b9acaf1429ebd5ca, user.id=60765e77836c4453991519c9216971f9, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:45.370914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.372335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.388220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 181/366] 10.222.0.61 () {56 vars in 1308 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/role_assignments?user.id=60765e77836c4453991519c9216971f9&scope.system=all&role.id=4f1be953ef494f47b9acaf1429ebd5ca => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.389336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.393397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc638-a7a9-4c87-8315-9d7e0b11370e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.409876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.436727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.437862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.438291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.438740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.440253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.442044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.444438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 186/367] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.446312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.539960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe83e2-c53e-46e1-ba86-7eac47dda21a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.618897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.625626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6067-8466-424f-990d-35c277880c79 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.672865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.673930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.674455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.675902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.676991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:45.677499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.679162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.690161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.690161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.690430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.690707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.698743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 187/368] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.698743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.723579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:45.734876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:45.735781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:45.736572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:45.742581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 182/369] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:45.742581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.781071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:45.872083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef8a2-fbd0-425f-b007-f7e207ee4d3b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.904522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57662-2080-40ed-834c-1481dce0b165 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:45.924892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.925558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.925843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.926121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.926964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:45.927277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.928944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.939332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 188/370] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.939992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:45.961286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:45.962385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:45.962872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:45.963543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:45.965107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:45.965958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:45.967852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:45.977481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 183/371] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:45.979373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:46.049741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.124976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ae9d-741d-4c43-9e4b-4480ebd6bc92 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:46.168298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:46.170296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:46.170889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:46.171399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:46.172402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:46.172940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:46.175137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:46.185979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 184/372] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:46.191938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:46.276927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569c4c0-34a3-4ea9-b814-9d51f606ead2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:46.299119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.315102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59054cc-6dda-47f7-ad77-0179d7b52751 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:46.333624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.334766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:46.335581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:46.335965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:46.336355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:46.359573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:46.360520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:46.361412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:46.361888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:46.438289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:46.440362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:46.518841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:46.520507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:46.552690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 185/373] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:46.562035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:46.584655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.706509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 189/374] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:46.715979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:46.728839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.746934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18a18-9ae6-4c9e-a3a0-c53900b739a0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:46.793403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:46.794556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:46.794850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:46.795135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] PATH_INFO: `/v3/projects/906e152e20d14899a1c568a7ddf7bb54` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:46.796610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] RBAC: Authorizing `identity:get_project(project_id=906e152e20d14899a1c568a7ddf7bb54)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:46.798141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:46.800209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 186/375] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:46 2022] GET /identity/v3/projects/906e152e20d14899a1c568a7ddf7bb54 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:46.807250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:46.910276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b28a5-317a-4ffa-827a-bdd486b7a90f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:46.957172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:46.979481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:46.981903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:46.984035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:46.984515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:46.987285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:46.989889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:46.994520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 190/376] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:46 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:46.995389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315c1d-ff0f-4a7f-b94b-5489b7d525c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:47.006060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:47.071695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:47.073383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.075245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.075769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] PATH_INFO: `/v3/groups/49b5c6f8854c487da016446ab00b2327` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.093428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] RBAC: Authorizing `identity:get_group(group_id=49b5c6f8854c487da016446ab00b2327)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:47.095301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:47.097789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 187/377] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:46 2022] GET /identity/v3/groups/49b5c6f8854c487da016446ab00b2327 => generated 229 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:47.103590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:47.161408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:47.194179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f44e-7d07-4125-94d5-6fa8ff487359 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:47.246811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720edd91-b7f8-49a5-a245-613bfd6227e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:47.260366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:47.261541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.263668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.264182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.265590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:47.268372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:47.270887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 191/378] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:47 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:47.278105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:47.314183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:47.316497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.317005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.318553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.319637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] RBAC: Adding query filter params (group.id=49b5c6f8854c487da016446ab00b2327, role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=906e152e20d14899a1c568a7ddf7bb54) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:47.321772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:47.323415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea3b87-2161-4846-8d59-50d550f2654b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:47.339262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 188/379] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:47 2022] GET /identity/v3/role_assignments?group.id=49b5c6f8854c487da016446ab00b2327&scope.project.id=906e152e20d14899a1c568a7ddf7bb54&role.id=6a9f236c49c24283afb390181f99e500 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:47.348444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bcfb8-a72b-4ea1-aa2e-ef3ba9e9b8d9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.349105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bcfb8-a72b-4ea1-aa2e-ef3ba9e9b8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.349579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bcfb8-a72b-4ea1-aa2e-ef3ba9e9b8d9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.351883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 189/380] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:47.361583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.362236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.362731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.378064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:47.405021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:47.406407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:47.451582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fa92-fd66-435c-ab39-432c76e5c8b4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:47.538439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:47.539817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.540146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.540367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.541250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:47.541558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:47.543020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:47.557596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 192/381] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:47 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:47.561106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 190/382] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:47.596835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:47.615232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:47.739953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f68680-6f69-4d71-a4db-105a38c422d7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:47.795182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:47.797296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:47.797296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:47.797296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:47.812465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:47.821308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:47.822281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:47.823091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:47.825823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 193/383] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:47.831288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:47.973270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de28c59-f020-418f-8a6e-3bf8b2123561 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:48.008842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:48.025447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:48.026209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.026506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.026790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.027567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:48.027880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:48.029392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:48.043326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 191/384] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:48.053789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:48.160205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:48.206801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23505a-6d02-4dc1-91b8-8b2be46b4584 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:48.256047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:48.256937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.257907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.258190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.265412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:48.266922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:48.268982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 194/385] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:48 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:48.274698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:48.413474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78e240-c23d-48c9-a4e5-10df490115ff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:48.458307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:48.459170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.459457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.459731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.461687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:48.463154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bce106-73bb-4ba7-b456-97bc0efb7fd6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:48.465107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 192/386] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:48 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:48.470521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:48.603627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c726a8-bda6-4ae8-abd8-88bcb7805466 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:48.650567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:48.651450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.651732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.652008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.658059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, group_id=6c579f8eb9dc4f5c92d6c494e650af9d, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:48.659538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078b117-ac92-492f-9251-71f9cade4468 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:48.662823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:48.676577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 195/387] 10.222.0.61 () {58 vars in 1442 bytes} [Thu May 19 21:47:48 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:48.712858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd3e84-e319-42f1-b65e-a5afa9dca968 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.713806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd3e84-e319-42f1-b65e-a5afa9dca968 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.714302 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd3e84-e319-42f1-b65e-a5afa9dca968 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.716324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 193/388] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:48.722282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:48.722624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:48.722905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:48.744336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:48.746344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:48.836432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 196/389] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:48.865986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd3e84-e319-42f1-b65e-a5afa9dca968 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:48.955759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:48.998707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd3e84-e319-42f1-b65e-a5afa9dca968 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.050985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.051782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.052066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.052343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.065338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6f88a74a-4c5c-4383-8888-2d65349f8245 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:49.074298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:49.075221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:49.076107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:49.076922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 194/390] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:49.083619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.084228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352e796-420b-454a-916c-d151312294b3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.084533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352e796-420b-454a-916c-d151312294b3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.084825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352e796-420b-454a-916c-d151312294b3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.087911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 195/391] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:49.092872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.093166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.093448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.119345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:49.119345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:49.223808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 196/392] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:49.248208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.262744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020e99-6d8c-45e1-985d-ca2ef645aded None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.290664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:49.300307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.301110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.301396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.301672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.302421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:49.302722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:49.304886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:49.308124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:49.315211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 197/393] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:49.325711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.363003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc2aa0-427a-451b-95b2-64427bfad727 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.404323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.405721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.406139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.406423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.421279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:49.433185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:49.434057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:49.434847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:49.436497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 197/394] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:49.440941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.450073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3915-4a97-4ddd-8d3b-3b86516cf4f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.492643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.493866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.494344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.494811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.496223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:49.497904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:49.504142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 198/395] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:49.505553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.575337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492190ce-4499-4fed-a58c-db06a1843a15 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.590447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:49.615539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:49.629738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.630655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.630890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.631165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.634373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:49.634688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:49.636088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:49.654455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 198/396] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:49.658840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.700949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89fb1a-01a7-4f49-b282-43b3ce145249 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.763540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.764322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.764604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.764903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.767174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:49.769982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:49.774883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 199/397] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:49.780994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.836834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab770-626f-4d45-b620-b97b4c1d1d73 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:49.877502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:49.878367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:49.878654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:49.878940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:49.880134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:49.881604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:49.884145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 199/398] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:49.889475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:49.935645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:49.998253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0035800-6167-4d30-a0f6-974a20261755 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.006862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f4b2-1847-4a3f-9869-c026c0dc0ac4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.037049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.037936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.038223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.038611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.040060 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:50.041579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:50.043492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 200/399] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:49 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:50.049381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:50.089696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:50.122796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.123693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.123979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.124257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.127809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, group_id=6c579f8eb9dc4f5c92d6c494e650af9d, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:50.129313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c51f88-576f-4b4b-a9eb-31be441e0c59 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:50.156974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 200/400] 10.222.0.61 () {58 vars in 1442 bytes} [Thu May 19 21:47:49 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:47:50.183859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd938aa-7826-497b-a6b3-f38ece84ff59 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.197036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:50.224534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.225475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.225764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.226199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.228792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] RBAC: Authorizing `identity:create_grant(project_id=2a90429a45174bfb9d1aa8661adbe36c, group_id=6c579f8eb9dc4f5c92d6c494e650af9d, role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:50.230527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1db866-009d-4bf7-ae84-355db310149c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:50.241517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf753-bb8f-4fd5-8372-3514edbb8077 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.242398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf753-bb8f-4fd5-8372-3514edbb8077 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.242663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf753-bb8f-4fd5-8372-3514edbb8077 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.248444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 201/401] 10.222.0.61 () {58 vars in 1442 bytes} [Thu May 19 21:47:50 2022] PUT /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:47:50.248939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 201/402] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:50.249904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c930-7e7f-47fa-9625-9e7c11500851 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.250206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c930-7e7f-47fa-9625-9e7c11500851 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.250493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c930-7e7f-47fa-9625-9e7c11500851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.282941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea04c930-7e7f-47fa-9625-9e7c11500851 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:50.284271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea04c930-7e7f-47fa-9625-9e7c11500851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:50.388583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 202/403] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:50.410961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf753-bb8f-4fd5-8372-3514edbb8077 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:50.444036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025627-322f-467f-8626-2955394eb206 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.444345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025627-322f-467f-8626-2955394eb206 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.444606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025627-322f-467f-8626-2955394eb206 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.446839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 203/404] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:50.453454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.453892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.454178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.483260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:50.484883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:50.571088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf753-bb8f-4fd5-8372-3514edbb8077 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.598227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 204/405] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:50.610496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.611339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.611618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.611884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.618552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:50.624456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:50.633064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:50.633940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:50.634694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:50.635653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 202/406] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:50.640129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:50.670421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:50.694487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:50.780646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe0b9-d6c0-450e-82c3-fdc49a77050b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.808137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0898172-e1f4-42d1-ae20-1a59ffbc548a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:50.852687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.853664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.854387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.854835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.855759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:50.856834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:50.858395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:50.864679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:50.870860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:50.871137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:50.871407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:50.880111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 203/407] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:50.883250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:50.889960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:50.898586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:50.899466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:50.900230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:50.900872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 205/408] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:50.909907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.001862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f605c-73e3-4f53-b86e-db8bb5eda503 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.010483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:51.051202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.052432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.052944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.053412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] PATH_INFO: `/v3/groups/49b5c6f8854c487da016446ab00b2327` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.055423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] RBAC: Authorizing `identity:get_group(group_id=49b5c6f8854c487da016446ab00b2327)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.057117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.059354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 204/409] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:50 2022] GET /identity/v3/groups/49b5c6f8854c487da016446ab00b2327 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:51.065773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.124901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:51.140212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5f62-3aaf-40f2-b41a-d6bf9b3210cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.203283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.203955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.204234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.204508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.205290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:51.205593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.206958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.221341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 206/410] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:51.227184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.246510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7f8f-c700-468f-9694-12756814eacd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.309969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.310878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.311167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.311444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] PATH_INFO: `/v3/projects/906e152e20d14899a1c568a7ddf7bb54` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.314571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] RBAC: Authorizing `identity:get_project(project_id=906e152e20d14899a1c568a7ddf7bb54)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.316087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.318739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 205/411] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:51 2022] GET /identity/v3/projects/906e152e20d14899a1c568a7ddf7bb54 => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:51.325672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.413071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:51.427859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a6d60-c9d5-4a88-8858-49960c994703 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.486156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af964c4-89e9-4c63-be8f-96f350df4f30 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.505381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.506552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.507031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.508108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.517568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.519660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be31130-7d2f-46fb-abe0-828191b2c489 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.520541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.521663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.521770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 207/412] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:51 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:51.521845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.522042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] PATH_INFO: `/v3/projects/906e152e20d14899a1c568a7ddf7bb54/groups/49b5c6f8854c487da016446ab00b2327/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.523292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d34-f1d0-42ae-a429-38aab3f2d276 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.524269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d34-f1d0-42ae-a429-38aab3f2d276 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.524559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d34-f1d0-42ae-a429-38aab3f2d276 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.524755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] RBAC: Authorizing `identity:create_grant(project_id=906e152e20d14899a1c568a7ddf7bb54, group_id=49b5c6f8854c487da016446ab00b2327, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.525949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 208/413] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:51.528653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.533993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.534305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.534753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.545511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 206/414] 10.222.0.61 () {58 vars in 1442 bytes} [Thu May 19 21:47:51 2022] PUT /identity/v3/projects/906e152e20d14899a1c568a7ddf7bb54/groups/49b5c6f8854c487da016446ab00b2327/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:51.546763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.557897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:51.559604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:51.576390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:51.665428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942712a9-6fa9-4688-908c-e02d80738309 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.666960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:51.673773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 209/415] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:51.703471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.704656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.705302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.705749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.707266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.708988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.712181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 207/416] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:51 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:51.714727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.717906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:51.831129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5aa08-4db3-4d32-933b-6b06d5a01da5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.873419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.875365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.875815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.876252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.876987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fc608-b96d-4afe-a83c-95e45b538f5e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:51.880473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] RBAC: Authorizing `identity:create_grant(project_id=0514cd744ba74b0a9abd0ddf8286aafe, group_id=6c579f8eb9dc4f5c92d6c494e650af9d, role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:51.882787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:51.901336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 208/417] 10.222.0.61 () {58 vars in 1442 bytes} [Thu May 19 21:47:51 2022] PUT /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe/groups/6c579f8eb9dc4f5c92d6c494e650af9d/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:47:51.924806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:51.925544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:51.925830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:51.926110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:51.938499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:51.947459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:51.948561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:51.949663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 210/418] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:51.957999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:52.004729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:52.107184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d674-3990-4853-b4d9-ac45079de57e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:52.143508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:52.144176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:52.144457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:52.144768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:52.145519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:52.145822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:52.147205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:52.160262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 209/419] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:52.161706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:52.176349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:52.276715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60c3c7-596f-4738-8813-6117302f8441 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:52.312837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:52.313865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:52.314353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:52.316261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:52.402829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:52.404699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:52.426136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:52.557084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 211/420] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:52.562358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:52.682176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:52.695769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed24bd5-8f25-46fa-9b5b-2bf3d2a8dcdb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:52.745451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:52.746329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:52.746612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:52.746889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:52.753348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:52.754836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:52.756919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 210/421] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:52 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:52.762595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:52.907784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7985d2-8b22-4c8b-9d2d-64fb88f1efe7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:52.919330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:52.944444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:52.945421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:52.945698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:52.945969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:52.947145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:52.948564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0d1da-8848-457a-b441-233d759e6cd3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:52.956520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 212/422] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:52 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:52.957910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.044400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80f68-79e3-472b-8846-e106d15cc4bf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.044724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80f68-79e3-472b-8846-e106d15cc4bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.045007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac80f68-79e3-472b-8846-e106d15cc4bf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.046360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 213/423] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:53.051684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8b38-1150-4595-a7d2-9127c8dad9c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.051967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8b38-1150-4595-a7d2-9127c8dad9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.052242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8b38-1150-4595-a7d2-9127c8dad9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.077174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a25c8b38-1150-4595-a7d2-9127c8dad9c2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:53.078360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a25c8b38-1150-4595-a7d2-9127c8dad9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:53.087703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e83bd8-69cb-4b0f-ae2c-e027fd8d5e0e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.154859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.155543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.155827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.156102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.157007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:53.157319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:53.159766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:53.176804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 211/424] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:52 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:53.238825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 214/425] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:53.276223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.307286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:53.340418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784a2aa-1b9b-45cc-983f-07a96c62710f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.340802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784a2aa-1b9b-45cc-983f-07a96c62710f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.341093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784a2aa-1b9b-45cc-983f-07a96c62710f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.342878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 215/426] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:53.348944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.349263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.349579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.388632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:53.389849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:53.403042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a11ce-0618-46c3-b5e2-3b629aeb5958 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.442001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.442775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.443144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.443473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.455030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:53.464819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:53.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:53.466492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 212/427] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:53.469279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 216/428] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:47:53.477608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.495141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.536278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:53.606511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:53.624704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab484-3612-4187-bdd0-f5f4095253c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.636782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10284a18-f612-410e-af12-fe3d67ce2253 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.674898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.675951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.676404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.676857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.688960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:53.692098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.692887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.693378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.693829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.694798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:53.695230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:53.696955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:53.699018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:53.700127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:53.701129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:53.703496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 217/429] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:53.710370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.714762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 213/430] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:53.721395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.855473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:53.861531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662863-4233-45e5-a838-c56f929275ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.863630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0df3bd-dfdd-45a7-b191-a8d117d8e025 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:53.907044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.907964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:53.908182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.908708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.909147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.909800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:53.910524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:53.911010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:53.911115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:53.911599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:53.913610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:53.926528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 218/431] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:53.932251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:53.936850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.014827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.018084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.080851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe1f10-7d7e-4a64-b0e9-a4b2ff7eaf7e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.130571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.131482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.131778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.133030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.139134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 214/432] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:47:54.146039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.169065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.197061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.224249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.225754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.256866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e19f-efc9-4b98-a67e-715cc0b8a8db None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.306100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.307607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.308457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.309237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.310809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.312744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.320662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 215/433] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:54.322370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.365966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 219/434] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:54.367744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.441207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.472821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484578-f919-4c90-b99f-16afb7af7d0b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.490635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.490922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2929a-fbec-4e21-a837-01d36b00e94c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.514277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.515139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.515425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.515702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.516918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.518386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4711b-cbeb-45d3-80b2-2a3a8465a41e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.519997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 216/435] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:54.521333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528323e-86f0-4a2c-9fbd-7fd0103721b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.521626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528323e-86f0-4a2c-9fbd-7fd0103721b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.521903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528323e-86f0-4a2c-9fbd-7fd0103721b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.524802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 217/436] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:54.529096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.529499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.529896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.532413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.533290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.533572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.533842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] PATH_INFO: `/v3/projects/2a90429a45174bfb9d1aa8661adbe36c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.535132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] RBAC: Authorizing `identity:get_project(project_id=2a90429a45174bfb9d1aa8661adbe36c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.536706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.542861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 220/437] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/projects/2a90429a45174bfb9d1aa8661adbe36c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:54.542861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.549152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:54.550322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:54.633199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 218/438] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:54.633199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.657340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe176b-5c9d-4d4a-9953-67df68946b66 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.687911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.698722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.699774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.700202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.700644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.701855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=c2ae6dbe1e314175bcda8fcf0e179597, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:54.702314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.704025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.711216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 221/439] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=c2ae6dbe1e314175bcda8fcf0e179597 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:54.712250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.761320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c281e-f87c-4f5c-9cb3-ff8e5af00ccd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.781396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.818111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.819293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.819847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.820294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.822482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:54.824783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:54.827945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 219/440] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:54.834611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.841828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227681ef-a6a9-45b2-83f3-e8f766a0bf94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.884389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:54.885142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.885495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.885776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.899277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bb8226cc-868d-4d3f-9972-721862e5553e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:54.907080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:54.908001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 222/441] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:54.909990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180dc2-a5e2-4cc7-a8c9-bfd9733f0d32 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:54.910683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180dc2-a5e2-4cc7-a8c9-bfd9733f0d32 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:54.910683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180dc2-a5e2-4cc7-a8c9-bfd9733f0d32 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:54.911531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 223/442] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:54.914606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180dc2-a5e2-4cc7-a8c9-bfd9733f0d32 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:54.964048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba65b-89c8-496f-a765-c8d51ca7dead None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:54.984881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.999689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.000350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.000646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.001034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.001868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=2a90429a45174bfb9d1aa8661adbe36c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:55.002176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.003676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2103e1-f591-4718-b017-b8a3d9b3ee67 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.010272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 220/443] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=2a90429a45174bfb9d1aa8661adbe36c&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.011429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.011744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.012165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.032751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:55.033992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:55.046813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180dc2-a5e2-4cc7-a8c9-bfd9733f0d32 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.081084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.090095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.090883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.091260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.091731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.092510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:55.092876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.094295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.105004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 224/444] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:55.110573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.113162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 221/445] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:55.137044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.233708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda30f2-f283-4dd7-bc6f-9f110368ec7a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.251870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.255097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f14ee-713a-4ad7-9bac-8d36ad0e47c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.278212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.279391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.279782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.280429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.292941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.293763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.294053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.295350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.306567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:47:55.316668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:55.317759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:55.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 222/446] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:47:55.322568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.366016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.370275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.371750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.444209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692a549-f717-4ed9-8115-96d4df77fc4c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.484984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.485666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.485953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.486342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.486995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:55.487306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.489732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.490591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 225/447] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:55.498060 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.499881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 223/448] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.505055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.513461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.609655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ccca5-caa9-40d0-9a2a-d4467da499f3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.610124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.617318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b6b86-5ff3-4191-9e24-c30a79c7440c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.645602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.646460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.646765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.647116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] PATH_INFO: `/v3/projects/906e152e20d14899a1c568a7ddf7bb54` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.648353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] RBAC: Authorizing `identity:get_project(project_id=906e152e20d14899a1c568a7ddf7bb54)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.649837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.651532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 226/449] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/projects/906e152e20d14899a1c568a7ddf7bb54 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.656833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.659112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.659759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.660043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.660321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.730011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.731420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.739614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.758795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5958f6-ed72-4271-97ca-0683a9b39f9b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.791451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.792291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.792564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.792866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] PATH_INFO: `/v3/groups/49b5c6f8854c487da016446ab00b2327` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.794028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] RBAC: Authorizing `identity:get_group(group_id=49b5c6f8854c487da016446ab00b2327)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.795448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.797245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 227/450] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/groups/49b5c6f8854c487da016446ab00b2327 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.802850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.824708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:55.831783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 224/451] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:47:55.837253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.901203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0717-dad4-4ff5-8bda-ce7ac5723ddf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.933998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37f30-eb82-4da5-a594-b996bd1c510d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:55.935329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.935975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.936259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.936534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.937466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] RBAC: Adding query filter params (group.id=49b5c6f8854c487da016446ab00b2327, role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=906e152e20d14899a1c568a7ddf7bb54) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:55.937780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.939200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.945110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 228/452] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/role_assignments?group.id=49b5c6f8854c487da016446ab00b2327&scope.project.id=906e152e20d14899a1c568a7ddf7bb54&role.id=6a9f236c49c24283afb390181f99e500 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.969177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:55.970063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:55.970346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:55.970622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] PATH_INFO: `/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:55.971876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] RBAC: Authorizing `identity:get_project(project_id=0514cd744ba74b0a9abd0ddf8286aafe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:55.973390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:55.975167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 225/453] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/projects/0514cd744ba74b0a9abd0ddf8286aafe => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:47:55.981832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:55.999604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:56.091847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7ca10-8069-4183-b9f3-db9d9f66a2c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:56.127249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:56.128110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:56.128386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:56.128706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] PATH_INFO: `/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:56.129881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] RBAC: Authorizing `identity:get_group(group_id=6c579f8eb9dc4f5c92d6c494e650af9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:56.131297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06296d9a-6965-4fba-ae0a-b16dd55c23fa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:56.133137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 229/454] 10.222.0.61 () {56 vars in 1180 bytes} [Thu May 19 21:47:55 2022] GET /identity/v3/groups/6c579f8eb9dc4f5c92d6c494e650af9d => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:56.138845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:56.201198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:56.239220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0390695-630d-4c82-8c83-edddd13561eb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:56.271245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:56.271883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:56.272163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:56.272435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:56.273881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] RBAC: Adding query filter params (group.id=6c579f8eb9dc4f5c92d6c494e650af9d, role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:47:56.274190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:56.275619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a58c9-717e-40da-96e9-3112ef383d9c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:56.281374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 226/455] 10.222.0.61 () {56 vars in 1376 bytes} [Thu May 19 21:47:56 2022] GET /identity/v3/role_assignments?group.id=6c579f8eb9dc4f5c92d6c494e650af9d&scope.project.id=0514cd744ba74b0a9abd0ddf8286aafe&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:47:59.448954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b6da-fee0-4d9a-8f8e-75f154335a3e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.449964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b6da-fee0-4d9a-8f8e-75f154335a3e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.450451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b6da-fee0-4d9a-8f8e-75f154335a3e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.452265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 230/456] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:59.470886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.471808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.472287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.497638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:59.499150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:59.529140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b7b-1fde-4ff1-9ab9-1f5aa9e2ba02 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.529898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b7b-1fde-4ff1-9ab9-1f5aa9e2ba02 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.530495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a31b7b-1fde-4ff1-9ab9-1f5aa9e2ba02 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.532606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 231/457] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:47:59.538606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021aa0-dc74-4f88-97e4-9f66961dc270 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.539284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021aa0-dc74-4f88-97e4-9f66961dc270 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.540250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021aa0-dc74-4f88-97e4-9f66961dc270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.560534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5c021aa0-dc74-4f88-97e4-9f66961dc270 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:59.562486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5c021aa0-dc74-4f88-97e4-9f66961dc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:59.607268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 227/458] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:59.654238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:59.661692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 232/459] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:59.663467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e50ab-fb3e-4765-86ae-ba3a490ee4a8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.663973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e50ab-fb3e-4765-86ae-ba3a490ee4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.664454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e50ab-fb3e-4765-86ae-ba3a490ee4a8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.666099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 233/460] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:47:59.678520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.679198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.679680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.703086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:59.704768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:59.721559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:59.724835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:59.770767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b60b1-f5bd-4fda-aa71-677425e793ff None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:59.791129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 234/461] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:47:59.792498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:59.810149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:59.811089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.811878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.811878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.824050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:59.825877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:47:59.845942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:59.846883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:59.847739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:47:59.847739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:47:59.847739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:47:59.847739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:47:59.847739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 228/462] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:59.848674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:59.920055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0b7c1-d949-47fe-b313-62b4c930ace2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:59.956870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:59.957174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a81053-7f0f-4eea-8867-fad2801f9a38 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:59.957584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.957869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.958149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:59.969433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:59.969679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:59.970900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:59.973026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-62ac43d3-552c-4c5a-bf66-8a2c7660b801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:47:59.995219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:47:59.996112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:47:59.997065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:47:59.997065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:47:59.997065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:47:59.997065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:47:59.997065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 235/463] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:47:59.998099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b385b-765a-4c3e-a6a4-51a8ca7cef7a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:59.998390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b385b-765a-4c3e-a6a4-51a8ca7cef7a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:59.998670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b385b-765a-4c3e-a6a4-51a8ca7cef7a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.000057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 236/464] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:00.000704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b385b-765a-4c3e-a6a4-51a8ca7cef7a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.005689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.006558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.006888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.007223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.021960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.024642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:00.047128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:00.048036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:00.048916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:00.048916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:00.048916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:00.048916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:00.053152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 229/465] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:00.054781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.120177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b385b-765a-4c3e-a6a4-51a8ca7cef7a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.161607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.162404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.162686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.162968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.164450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:00.164783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.166369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041b23-d761-446f-8669-fb5efd8b2e5d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.166697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627661-51ff-42eb-8fd6-0dbe99565b1f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.178927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 237/466] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:00.180445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.180822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.181109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.198119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:00.199312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:00.214040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.214903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.215188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.215465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.216354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:00.216972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.218355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.238676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 230/467] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:00.238676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.251266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.265807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.274831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 238/468] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:00.275990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.350893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccb59-0401-45dc-acba-7572b3544a5e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.386715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.387414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.387734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.388046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.388901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:00.389252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.390754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.395456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69a2fc-e359-496c-83a2-739f97942703 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.404088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 231/469] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:00.404698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.432863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.433524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.433925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.434205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.434905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.436330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.456147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 239/470] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/users => generated 248 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:00.457420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.509436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.511942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.534263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67835e-def5-478d-9390-e02178f53946 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.572658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.573384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.573678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.573964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.574717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.576218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505c8cb-4e27-4273-88af-a0c2c14d4b06 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.580842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b358-2002-405e-ac04-6cdf7b825b13 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.597727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 232/471] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:00.599181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfa826-dfcd-4e44-b040-24937f7146f5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.599546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfa826-dfcd-4e44-b040-24937f7146f5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.599897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfa826-dfcd-4e44-b040-24937f7146f5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.602246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 233/472] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:00.602882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfa826-dfcd-4e44-b040-24937f7146f5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.651465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.651902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.652773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.652773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.668014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.669578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a5a3ccaf-0bf4-4bca-a94e-70b724af07c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:00.696112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:00.697037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:00.697850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:00.697850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:00.697850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:00.697850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:00.700216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 240/473] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:00.701966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.702487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.702959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.722960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:00.724514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:00.769703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfa826-dfcd-4e44-b040-24937f7146f5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:00.816155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 241/474] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:00.817348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.820882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:00.821618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:00.821913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:00.822253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:00.822973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:00.824568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:00.836151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.853159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:00.859438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 234/475] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/users => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:00.860230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:00.979285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbc525-da50-4dfd-8f6d-2328ace76088 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:01.027092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83308a6-b344-41a9-adf2-57811c8a845e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:01.054443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:01.055244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:01.056331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:01.056331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:01.056331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:01.056800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:01.058071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:01.078287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 242/476] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:01.083897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:01.089347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:01.090079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:01.090643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:01.090924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:01.091697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:01.094118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:01.103838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:48:01.103838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 21:48:01.103838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [parameters: {'id': '7d7779000213489c98c029a010e94741', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 21:48:01.103838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() May 19 21:48:01.126651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return ref.to_dict() May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:01.127566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:48:01.128470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:48:01.129352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:01.130236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:01.131049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': '7d7779000213489c98c029a010e94741', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:01.131805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 21:48:01.135457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.139686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 235/477] 10.222.0.61 () {60 vars in 1131 bytes} [Thu May 19 21:48:00 2022] POST /identity/v3/roles => generated 144 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:01.139686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:01.205949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:01.218759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:01.232837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1fee-db79-4692-a9b9-d42783248941 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:01.274671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:01.275347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:01.275626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:01.275899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:01.276297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e62d-664c-4aed-9f11-6cbd4b06b79d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:01.276682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:01.278325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:01.305005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 243/478] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:01 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:48:01.333890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:01.334742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:01.335069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:01.335356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:01.348708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-60a6e634-7a05-4ed1-a680-f352f895543c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:01.357800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:01.358787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:01.360028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 236/479] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:01.365323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:01.504243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76f0e1-ac5f-45e0-b59a-3acdef208e81 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:01.548890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:01.549574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:01.549857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:01.550135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:01.550890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:01.551194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:01.552549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4568bc-108a-4bc3-9b78-a8c39fc6a3aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:01.558714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:01.563707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 244/480] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:03.355002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0d202-9576-479d-9aa7-9fe68dc7f619 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.355513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0d202-9576-479d-9aa7-9fe68dc7f619 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.355689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0d202-9576-479d-9aa7-9fe68dc7f619 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.357235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 237/481] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:03.362581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e3aa-da85-4321-a65e-29b08b58ac4b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.362997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e3aa-da85-4321-a65e-29b08b58ac4b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.363232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e3aa-da85-4321-a65e-29b08b58ac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.380634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1125e3aa-da85-4321-a65e-29b08b58ac4b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:03.381880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1125e3aa-da85-4321-a65e-29b08b58ac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:03.456723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 245/482] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:03.474479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0d202-9576-479d-9aa7-9fe68dc7f619 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:03.593563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93a7c-777b-4105-90d3-1269b89126c6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.593906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93a7c-777b-4105-90d3-1269b89126c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.594128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93a7c-777b-4105-90d3-1269b89126c6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.596099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 246/483] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:03.596463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0d202-9576-479d-9aa7-9fe68dc7f619 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:03.605614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.606170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.606659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.624016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:03.625593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:03.641838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:03.691201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:03.692578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.693127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.693624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.703796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 247/484] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:03.713528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:03.721721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:03.723772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:03.724778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 238/485] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:03.730591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:03.740471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:03.798660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:03.855727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722e0d3-6663-491d-bc51-6d49630d6c94 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:03.884148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6166f-bf14-42c7-aed4-596a8b31aa55 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:03.904298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:03.905425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.905958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.906448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.907396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:03.907923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:03.909741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9081b59-bf99-4ce8-8b4f-41667b776dcd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:03.919854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:03.920688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.921111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.921236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.921470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 248/486] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:03.923014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fed-40b9-4a3a-93cb-fa680089c997 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.923501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fed-40b9-4a3a-93cb-fa680089c997 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.923972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fed-40b9-4a3a-93cb-fa680089c997 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.925771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 249/487] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:03.926673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f047fed-40b9-4a3a-93cb-fa680089c997 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:03.937129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b72f90a6-a4a0-4d00-9299-59ff41b0f59b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:03.945837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:03.946910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 239/488] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:03.949867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:03.950591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:03.950591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:03.968783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:03.970079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:04.002587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.050249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f047fed-40b9-4a3a-93cb-fa680089c997 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.057940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 240/489] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:04.058792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.077147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.097271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.098144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.098486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.098812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.112325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.114074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:04.134693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:04.135588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:04.136434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:04.136434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:04.136434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.136434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:04.138952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 250/490] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:04.139992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.195329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2035b1-cab9-4519-a66d-a95b66432c5e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.243817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.245020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.245863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.246292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.247204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:04.247657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.249804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.262007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 241/491] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:04.262405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448dfb-c0dc-414d-99e1-8dc665ff50c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.263269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.317266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.318724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.319387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.319915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.320115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.338046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:04.353398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:04.355626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:04.357641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:04.359784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 251/492] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:04.361289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.411822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.429304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fe266-8663-40d9-84fc-65251cbe6a17 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.462837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.463575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.463863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.464145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.464904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:04.465217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.466612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61a6a5-0b77-4f4f-9af2-ca789f056267 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.478679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c4ca4-23cd-47c6-9207-2fa8847024f3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.479729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 242/493] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:04.481947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e136-4d66-4ae8-b585-a94dc609e41e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.482266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e136-4d66-4ae8-b585-a94dc609e41e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.482549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e136-4d66-4ae8-b585-a94dc609e41e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.483996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 243/494] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:04.484765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9e136-4d66-4ae8-b585-a94dc609e41e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.513770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.514559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.514844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.515125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.526576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.528017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d05349c4-eeed-40bb-8b23-5b26d067d0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:04.545118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:04.545985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:04.546806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:04.546806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:04.546806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.546806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:04.546806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 252/495] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:04.547529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a4b3-721c-492d-8475-2d9c4e783bf4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.547763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a4b3-721c-492d-8475-2d9c4e783bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.548039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a4b3-721c-492d-8475-2d9c4e783bf4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.549315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 253/496] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:04.550010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9a4b3-721c-492d-8475-2d9c4e783bf4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.579748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.590491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9e136-4d66-4ae8-b585-a94dc609e41e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.626703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.627765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.628237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.628750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.630081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:04.630599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.633643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69791fdd-52f0-40fe-93d7-d29cd67d6629 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.637589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.641838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 244/497] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:04.643102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06961-1923-410c-8a73-32446aea8e40 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.643390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06961-1923-410c-8a73-32446aea8e40 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.643663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06961-1923-410c-8a73-32446aea8e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.653946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9a4b3-721c-492d-8475-2d9c4e783bf4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.659480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ccd06961-1923-410c-8a73-32446aea8e40 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:04.660684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ccd06961-1923-410c-8a73-32446aea8e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:04.688223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.688976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.689282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.689563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.697041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.698477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:04.713560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:04.714479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:04.715371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 254/498] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:04.715966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.730800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 245/499] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:04.732127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.732418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.732726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.756610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:04.758512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:04.816198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98f04-0082-4bd3-9bce-df75ba06212d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.824822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 246/500] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:04.825639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.840071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.863068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.863725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.864012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.864288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.865046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:04.865353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.866748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.878468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 255/501] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:04.879256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:04.888130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:04.926830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9eec-3960-4398-a026-e0cde2adc616 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.962309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:04.963040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:04.963327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:04.963607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:04.973544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:04.975017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:04.977907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec352-f4e9-459d-8485-3b3fbf6ba7f1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:04.993829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:04.994852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:04.995901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:04.995901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:04.995901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:04.995901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:04.995901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 247/502] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:04.996215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.014728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.015397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.015690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.015976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.016841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:05.017163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.018604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.056069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 256/503] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:05.056936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.059845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.099479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abd9e7-24f5-4c07-bdee-c1b805f8fcc1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.118965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.134516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.135312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.135596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.135870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.145948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.147559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.157129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145db0-e299-4e2e-9042-092a4286ad20 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:05.165259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:05.166105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:05.166954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:05.166954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:05.166954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:05.166954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:05.166954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 248/504] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:05.167222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.195124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.195916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.196198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.196498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.206708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:05.214589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:05.215488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:05.216332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 257/505] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:05.217072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.270830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639400-bc72-40b9-828d-9d42e5317e49 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.287627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.306458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.307257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.307543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.307823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.314842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.316203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f2a4-9bb6-41fa-803b-91dbc6a7f602 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.316301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:05.332813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:05.333724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:05.334594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 249/506] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:05.335350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.345159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.357188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.357846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.358132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.358409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.359135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:05.359445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.360839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.374081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 258/507] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:05.374879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.436016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358e4d-4128-4527-846e-b204df8dd859 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.470148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.470863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.471151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.471432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.475687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc1fc9-a1bf-48a4-98fc-68cc4c0db67a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.510225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.510879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.511169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.511451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.512174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:05.512490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.513906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.515729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.526153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 259/508] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:05.526779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.543599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.553372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.571154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.625918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c5ccf-bb58-4472-879f-38c900b752b2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.659606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.660267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.660551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.660862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.661604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:05.661917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.663317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.663960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 250/509] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:05.664846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.675155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 260/510] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:05.676016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.748328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.771608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdc567-7461-4acf-9f70-814a30ed93d6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.784379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978277a-1e8f-4693-9b1d-740d6acb908b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.791377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.806511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.807181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.807469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.807752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.808597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:05.808941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.810396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.818663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.820117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.820545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.820877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.828327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.829799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:05.843908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:05.844811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:05.845739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 261/511] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:05.846337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.850546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 251/512] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:05.850546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:05.948609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6bc91-f794-4be6-8c6e-2043ff8fa342 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.949441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da672d-2ff4-4dca-8394-72e10db3a7dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:05.983016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.983823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.983921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:05.984094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.984375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.984516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:05.984846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:05.985124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:05.985827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.986366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:05.987262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:05.996344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:05.998417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.009294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 262/513] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:05 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:06.010406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:06.018185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:06.019105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:06.020412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:06.020412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:06.020412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.020412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:06.020412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 252/514] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:06.020893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.049861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.137275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf83de0-cf7b-4a2e-be2d-97aeded3b08a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.143834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f9d2a-a01e-4d3f-8873-5a23f6baf152 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.176387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.177658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.178539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.179807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.181013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:06.181600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.183354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.183840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.184688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.185052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.185557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.199856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.201513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.202097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 253/515] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:06.206732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:06.224905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:06.226598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 263/516] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:06.228429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.262677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.314453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.321386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26741b-fe77-4802-8dc9-b6dbd0598bdb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.344926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000097-1b1d-4a65-bd8c-3745ace82c46 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.359438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.360096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.360379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.360680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.386588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.388064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.388872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.389375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.390814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:06.391524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.393410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.406653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 264/517] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:06.407406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.442481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.443991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.494377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.516272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf6c7f-8827-403f-95c3-297f43e91981 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.556591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.556811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 254/518] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:06.556904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.556904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.557076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.557155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.559898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.630479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.632577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.660061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371829b1-7752-4d26-9fc1-f461ea2a9680 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.695529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.696193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.696478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.696787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.697515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:06.697844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.699238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.715460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 255/519] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:06.715460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.716310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.746883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 265/520] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:06.746883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.784307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:06.826216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca06ab-9ac1-413f-912b-e3a8eaf08cea None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.856516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935a849-a7b9-4221-b0c4-9c6e67b33fef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:06.861976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.862775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.863063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.863343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.870567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.872030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:06.887847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:06.888930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 256/521] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:06.894666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.901351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:06.902273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:06.902560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:06.902839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:06.916379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:06.917921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:06.935801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:06.936881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 266/522] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:06.942517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:06.969916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.012386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49097fbe-3e5b-4a40-8818-ad2f5057af61 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.031185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.048550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def8ec4-42e3-438c-a643-0d2f1e9871c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.055795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.056717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.057065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.057804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.066386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.067929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:07.083197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:07.084265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:07.085538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:07.086342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 257/523] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:07.087766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.088639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.089015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.089361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.091592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.102382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.104548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:07.122738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:07.123808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:07.124798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:07.124798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:07.124798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:07.124798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:07.124798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 267/524] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:07.125225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.206147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ada4c-6ad9-4ddd-aeb4-f3f4cc13b6a4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.233226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.236939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a163d32-78c9-4778-8247-a9ad4e4d0147 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.246329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.247218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.247693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.247969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.248797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:07.249110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.250575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.270926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 258/525] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:07.271599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.276198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.278097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.278598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.279101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.280047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:07.280557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.282683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.292949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.300506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 268/526] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:07.301233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.392016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef9cd-17cc-43f9-9630-5fa201cb43f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.405847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd383-d3bd-4c88-8d4e-c335c85c1bdd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.431031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.431966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.432254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.432715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.433889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:07.434350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.435828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.440818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.441473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.441755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.442031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.442745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:07.443050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.445612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.454930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 259/527] 10.222.0.61 () {56 vars in 1139 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:07.455744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.466721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 269/528] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:07.466721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.506615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.528813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.570117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2c04d-91a1-45cb-992b-646256a0f2f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.575636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4cbd0-8dc5-48e5-8715-a5ade4bf006b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.605213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.605860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.606142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.606417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.607242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=41cfa205823a4c29abea3001bf086711) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:07.607551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.609015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.609832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.610624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.610915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.611193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.620674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.620674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.623109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 260/529] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/role_assignments?user.id=41cfa205823a4c29abea3001bf086711&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:07.624012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:07.639122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:07.640025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:07.640900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:07.648769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 270/530] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:07.650075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.746891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ba9a-c96d-4b9f-b9a8-9be523cf3cfc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.754782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.781413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c613ed-594c-49c8-8850-45f7795b8c09 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:07.787066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.787732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.788694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.789188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.806658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:07.828243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:07.830364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:07.830862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:07.831379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:07.841389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.844127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:07.865838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:07.867524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:07.868357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:07.874658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 271/531] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:07.874658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:07.875498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:07.877040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.007922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d05e-825c-407a-a685-b5d13c69b70d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.012429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 261/532] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:08.013541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.046846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.050773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.051569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.051953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.052336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.053338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:08.053806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.055384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.075206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 272/533] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:08.076427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.124159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.155543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e72c-fe73-4c3a-864e-03d2c31eea77 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.199276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.200050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.200399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.200791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.201642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:08.202016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.203904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.214792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31a56-6585-44ca-8ee0-1a0775975e57 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.216825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 262/534] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:08.218463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.262127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.263865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.264336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.265279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.279957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.282339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.290285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:08.308238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:08.309136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:08.309967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:08.309967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:08.309967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:08.309967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:08.312704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 273/535] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:08.313800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.338115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce38bc-8131-4653-917b-3c3f15904931 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.375108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.375683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.375967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.376260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.377092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=de01947b95e746f8a5d6f711d982f97e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:08.377406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.378850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.389009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 263/536] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/role_assignments?user.id=de01947b95e746f8a5d6f711d982f97e&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:08.389009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.433312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.445847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d08c-7f9a-4026-a475-b70cdd75a5ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.489808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.490735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.492056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.492784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.494579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:08.495303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.497697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218bc36-0c9f-4001-be10-0f916d5dcdae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.499242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c7e5d-a5af-4506-b3fd-b117844eff6f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.505486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 274/537] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:08.507982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31ff3-1b23-4706-93bd-7fd8398baecd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.508430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31ff3-1b23-4706-93bd-7fd8398baecd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.508898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31ff3-1b23-4706-93bd-7fd8398baecd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.510934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 275/538] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:08.516688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.516962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.517343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.535410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.543767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.544832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.545264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.545700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.546632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:08.547416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.550627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.572653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:08.572653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:08.573353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 264/539] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:08.575777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.675309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 276/540] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:08.707007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2964e8-d547-4842-9d98-6579b7eee60b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.730397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.771364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.771364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.771364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.771757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.780296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:08.782072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:08.800300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:08.801424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:08.802454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:08.808728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 265/541] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:08.810562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.843284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.862575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664ec23-e002-4538-8ecf-52ab3486f9c4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:08.907760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:08.914202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:08.915032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:08.915320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:08.915601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:08.935574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:08.943285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:08.944915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:08.945892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:08.957179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 277/542] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:08.963155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:08.964378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34470b0-debb-4667-8702-b1f78d4664da None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.024953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.025630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.025919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.026200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:09.027010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:09.027320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:09.028817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:09.061050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 266/543] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:09.064011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:09.179322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c684ef6-bad2-4a93-a873-57ddd8115932 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.221991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69efd-5c6a-4402-9f63-14ac32624355 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.238011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:09.255025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.256033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.257089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.258254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:09.259235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:09.260785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:09.262384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:09.280890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.281898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.282957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 278/544] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:09.283445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.283921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:09.285012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=5cfd4c34ecd84abe8310817755ae62fa) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:09.285526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:09.287321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:09.289055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:09.305048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 267/545] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:09 2022] GET /identity/v3/role_assignments?user.id=5cfd4c34ecd84abe8310817755ae62fa&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:09.305721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:09.411482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d6b41-3f96-4e4f-8c18-9e1622670d94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.452477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.453653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.454136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.454611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:09.467664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:09.469622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:09.490331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:09.492061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:09.492978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:09.492978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:09.492978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:09.492978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:09.495446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 279/546] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:09.501745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:09.664378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ac17-40cc-4568-ab90-16f84e837d1f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.716816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143199-1978-4920-959d-f58396e0065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.717486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143199-1978-4920-959d-f58396e0065e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.717859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143199-1978-4920-959d-f58396e0065e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.718087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143199-1978-4920-959d-f58396e0065e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:09.718953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb143199-1978-4920-959d-f58396e0065e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:09.719342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb143199-1978-4920-959d-f58396e0065e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:09.720918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb143199-1978-4920-959d-f58396e0065e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:09.742688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 268/547] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:09.747645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:09.871091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:09.944468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66bd4c-3735-445c-b375-080d73e93968 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:09.994459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:09.995178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:09.995471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:09.995749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.007641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:10.008828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8ab9737c-11ea-4fcd-833f-80d946ef8092 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:10.029393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:10.030728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:10.031800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:10.036413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 280/548] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:10.036533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143199-1978-4920-959d-f58396e0065e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:10.093855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30db5d-2599-4057-a223-3ee9a71aab23 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.094172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30db5d-2599-4057-a223-3ee9a71aab23 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.094392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30db5d-2599-4057-a223-3ee9a71aab23 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.097349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 281/549] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:10.101840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.102142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.102414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.110866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:10.129175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:10.130387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:10.231760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143199-1978-4920-959d-f58396e0065e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:10.250820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 282/550] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:10.277514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:10.278196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.278493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.278774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.279624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:10.279937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:10.281418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:10.288976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:10.312835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 269/551] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:10.321270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:10.380989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:10.485232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:10.487819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b2d35-21a9-466a-ba4f-5c08ddf8f77d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:10.508063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067720e-aac9-4596-8056-702bf1b83859 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:10.561309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:10.562356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.562834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.563301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.581344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:10.582167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.582459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.582744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.625250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:10.633198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:10.634188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:10.634996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:10.640413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:10.641047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 283/552] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:10.642482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:10.646634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:10.761169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:10.773338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 270/553] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:10.778842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:10.867809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb45a-cc6e-4964-853f-645c6d20c1ad None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:10.877797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:10.909366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:10.910530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:10.910824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:10.911101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:10.912035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:10.912373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:10.914479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:10.925146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 284/554] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:10.931144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.017183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a6917-1865-4bf9-bdc6-6891fe060131 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.061374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.062323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.062612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.062891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.076337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.077881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-93e3c4b6-9752-4f66-8fab-d4d096a78c8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:11.103165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.104365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.105280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:11.105280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:11.105280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:11.105280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:11.105993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 271/555] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:11.106197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0440c-5f4b-4bf7-8547-453490cb23ab None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.108638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595b93-fbcf-44e4-9632-7727838eac9e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.108638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595b93-fbcf-44e4-9632-7727838eac9e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.108638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595b93-fbcf-44e4-9632-7727838eac9e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.110704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 272/556] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:11.111502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6595b93-fbcf-44e4-9632-7727838eac9e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.158377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.159172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.159458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.159733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.169398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.176800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.180307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5eb8332c-7038-49c0-8dd8-e1bd2e09d594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:11.204400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.205372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.206788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:11.206788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:11.206788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:11.206788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:11.209129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 285/557] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:11.211609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1db4-4b02-43f2-b09a-bdc44353e119 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.212076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1db4-4b02-43f2-b09a-bdc44353e119 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.212517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1db4-4b02-43f2-b09a-bdc44353e119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.230548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e99d1db4-4b02-43f2-b09a-bdc44353e119 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:11.232123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e99d1db4-4b02-43f2-b09a-bdc44353e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:11.242901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6595b93-fbcf-44e4-9632-7727838eac9e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.251752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.287133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.288053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.288713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.288812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.290045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:11.290556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.292406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.308772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 273/558] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:11.309964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.312510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 286/559] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:11.314186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3171b-28f9-4a67-b980-ea51d4c3504f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.314704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3171b-28f9-4a67-b980-ea51d4c3504f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.315209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3171b-28f9-4a67-b980-ea51d4c3504f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.317360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 287/560] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:11.318319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3171b-28f9-4a67-b980-ea51d4c3504f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.424582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862519-4b9a-434d-b02a-1cd57922ba84 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.426085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3171b-28f9-4a67-b980-ea51d4c3504f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.427847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.462814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.463613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.463902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.464185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.464818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.465723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.466375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.466832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.468007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:11.468508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.470095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7ff4e-3477-465f-96a1-86fd9114fba5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.471315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.472776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.485822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 274/561] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:11.487405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:11.487845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.488735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:11.489610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 288/562] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:11.490378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.490544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.491120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.509870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.514018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:11.515621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:11.588736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 275/563] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:11.589525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.621457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e42b-140f-4163-a2a5-34c41c180e5d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.658011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.659050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.659335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.659610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.670544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:11.678214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.679213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:11.680035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 289/564] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:11.681221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.700769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.708928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621dd3-5c24-4892-8b9c-4e9012b13c56 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.744802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.745467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.745746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.746017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.746803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:11.747259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.748733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.753790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.768660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 276/565] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:11.768660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.787002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f1482-1c81-4dd5-8cd5-ac7ae9ad108c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.827521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.828739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.829225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.829609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.838282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:11.839920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:11.856989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.858062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 290/566] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:11.864248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.885640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191e174-16d2-416b-b1e4-3d77f36718a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:11.923209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:11.924010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:11.924323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:11.924602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:11.936559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:11.944644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:11.945759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:11.946581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 277/567] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:11.947250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:11.967618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:11.977231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c7c93-1a30-43ab-9b9f-179c4faa93ee None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.021401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.026646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.027773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.028297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.028975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.031909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.031909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.031909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.042894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 291/568] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.044336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.056919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bc879-0ca0-4847-babb-394dbc2328db None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.090891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.091476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.091765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.092046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.092915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=1d1c9cf9b02a41f39f46e878747cea3f) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.093232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.094691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.100091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 278/569] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:11 2022] GET /identity/v3/role_assignments?user.id=1d1c9cf9b02a41f39f46e878747cea3f&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.101087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.155801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b179-39bf-4a95-858d-551d57c6af26 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.197637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.198328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.198618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.198897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.199758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.200075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.201646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9dc51-3efc-4b95-9bdf-ee988018b9d7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.206489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.221674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 292/570] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.227724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c374a2-e3ed-4b6a-8256-3254ab0c246f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.228020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c374a2-e3ed-4b6a-8256-3254ab0c246f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.228305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c374a2-e3ed-4b6a-8256-3254ab0c246f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.229982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 293/571] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:12.230710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c374a2-e3ed-4b6a-8256-3254ab0c246f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.239422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915d6-478b-48ce-92e7-858e15da49de None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.286098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.287074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.287350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.288212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.288960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.289892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.290642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.290758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.299946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 279/572] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.302313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.347948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c374a2-e3ed-4b6a-8256-3254ab0c246f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.386171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.386969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.387261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.387543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.398575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.400018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.418004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb751be-3b79-4125-bb07-6f7098067feb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8a9da7ef-69b7-4f74-8e70-16ce4451152c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:12.419727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:12.420703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:12.421587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:12.421587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:12.421587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.421587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:12.425731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 294/573] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:12.426962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.427258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.427544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.445117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:12.446363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:12.457803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.458596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.458992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.459279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.468963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.472063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.473882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:12.495147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:12.496183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 280/574] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:12.497445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.532642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 295/575] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:12.532642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.567222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.618926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be41c-cec5-42c9-8560-4f8ad1bac8a0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.640120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371a4ef-43d8-4d58-9187-31081d5122ed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.654993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.655849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.656139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.656448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.668724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.670201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.680686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.681695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.682164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.682629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.683537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.684026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.685606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:12.688383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:12.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:12.690455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:12.690455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:12.690455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:12.690455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:12.690455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 281/576] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:12.690868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.706657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 296/577] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.706657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.743467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.805577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906162-4e3e-436e-9c80-f160ff16cac3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.811037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:12.818506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483ef0a-11e8-4f85-9a48-58e8e343fdb2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.842315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.842990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.843276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.843561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.844290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:12.844599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:12.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:12.855840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:12.856655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:12.856954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:12.857232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:12.860280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 282/578] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:12.861313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.870926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:12.878642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:12.879562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:12.880402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 297/579] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:12.881131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:12.975673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe3ff2-768b-4fe2-b733-ad12d5ac6050 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:12.979627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.001161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955134f6-c88e-4c5a-abee-314ee9621667 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.021535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.022407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.022407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.023009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.023651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:13.023954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.025388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.038292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 283/580] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:13.039191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.052490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.054544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.054973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.055529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.064438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.065920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:13.080300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:13.081461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 298/581] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:13.087442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.102775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.151552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a338a-27eb-4173-8799-1b61ddaced89 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.187840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.188706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.189000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.189280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.197399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.198871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.203589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3aae6-771b-4585-ac1e-f4bf6e53ecc5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:13.214832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:13.216011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:13.217151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:13.222790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 284/582] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:13.222790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.247626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.249473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.250231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.250609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.251052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.252076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:13.252391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.253789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.267982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 299/583] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:13.267982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.334411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00356c-67a2-4684-bb19-8415cd2b926d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.356848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.373159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.373954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.374234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.374506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.383900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674772-bc2f-48be-a857-8631ee4038ed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.385080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.386550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:13.402542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:13.403567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:13.404651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:13.408532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 285/584] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:13.409388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.425251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.426185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.426734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.427316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.428757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:13.429157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.430584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.448646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 300/585] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:13.449705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.502987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.521113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2835f-01cc-469a-b6cf-89b9282cea09 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.555066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5315619-df1e-4f65-a3d9-5ee54ace8efd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.559900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.560557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.560864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.561139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.561916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:13.562215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.564992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.578038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 286/586] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:13.579165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.597805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.599019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.599621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.599994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.602143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.612385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.614715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:13.639172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:13.640094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:13.640983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:13.640983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:13.640983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:13.640983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:13.641710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 301/587] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:13.642186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.689864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad5a9f-e73b-4093-9157-7753011052a8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.726953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.728021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.728547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.729285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.730340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:13.730710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.732221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.752676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 287/588] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:13.753346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.756185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.769769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce30795-3034-4530-846c-6660036f96ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.807022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.807820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.808108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.808379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.820072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.821615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:13.839649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:13.840887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:13.841903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:13.841903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:13.841903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:13.841903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:13.841903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 302/589] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:13.842286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.867730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b063cd5-d609-4a7b-9d14-dd53bbd506e9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:13.887768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:13.905454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:13.906504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:13.906892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:13.907175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/41cfa205823a4c29abea3001bf086711/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:13.923585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=41cfa205823a4c29abea3001bf086711, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:13.925100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:13.938756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 288/590] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:13 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/41cfa205823a4c29abea3001bf086711/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:13.939564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:13.965914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcb875-78da-406f-b59a-c9ff07c26311 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.001228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.006104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.009060 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.009702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.010135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.011181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:14.011696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.013351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.040467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 303/591] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:14.041673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.064477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde80b-1325-453e-8815-aeb5118048f4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.107817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.108762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.109093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.109369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.121769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.123426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:14.143438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:14.144655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:14.145772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:14.145772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:14.145772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:14.145772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:14.149891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 289/592] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:14.150845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.165485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.173197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c119665-3130-441b-8b2c-6c59db9d7925 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.218195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.218798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.219082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.219359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.220103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:14.220410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.222678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.241893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 304/593] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:14.243055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.291512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdbee-132a-4204-a51c-b13f1194b6bf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.295217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.332658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.333808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.334292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.335299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.344264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.346100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:14.364343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:14.365762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:14.366986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:14.369400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 290/594] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:14.374022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881714a-b592-4bc0-8b83-e1b16cc8f8fa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.375180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.418853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.419894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.420371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.420902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.422004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:14.422531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.424466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.442132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 305/595] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:14.442132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.462324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.492188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a924c-b83b-435d-a9bd-4c56a699e9c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.528841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.529882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.530288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.530700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.539046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.540768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.555150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b62ae4-832b-4234-a858-1b02a96fdc8c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c4f9b35c-9fa0-4f12-8bd0-e2141c129478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:14.557327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:14.558363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:14.559455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:14.560479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 291/596] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:14.561736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca3bc3-080c-4880-9349-1d199ec6c2b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.562026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca3bc3-080c-4880-9349-1d199ec6c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.562310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca3bc3-080c-4880-9349-1d199ec6c2b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.563843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 292/597] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:14.564460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca3bc3-080c-4880-9349-1d199ec6c2b6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.596091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.598451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.599289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.599653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.600546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.602045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:14.602361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.603784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.627249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 306/598] 10.222.0.61 () {56 vars in 1139 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:14.627249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.686336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca3bc3-080c-4880-9349-1d199ec6c2b6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.713869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.726208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.727513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.727831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.728142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.738580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.739808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687105de-cb8a-4905-a059-439e890db7e7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.740215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b45b6629-ddc3-49f9-a450-cb43235efaa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:14.761155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:14.762226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:14.763155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 293/599] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:14.764054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.764369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.764713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.780959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.781615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.781900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.782180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.782958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:14.783269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.783947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:14.784816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.785312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:14.800818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 307/600] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:14.806166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.862181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 294/601] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:14.863085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:14.889964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.926521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9a20b-d19d-4fdd-baff-dba83d82acbb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.963424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:14.964281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:14.964648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:14.964992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:14.968281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:14.978613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:14.980514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:14.981899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81931c1-679a-4b48-a300-a5d2679a3db3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:14.999268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:15.000359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:15.001602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:15.001602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:15.001602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:15.001602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:15.001602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 308/602] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:15.003042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.031608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.032764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.033151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.033908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.035087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:15.035742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.039176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.056781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 295/603] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:15.057620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.115447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1979c2-cbe8-458e-af42-088317f6dccd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.144134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:15.155089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.156163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.156454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.156774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/de01947b95e746f8a5d6f711d982f97e/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.173920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f86d2b-f0e3-498a-95b4-b54c878243f0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.176298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=de01947b95e746f8a5d6f711d982f97e, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.177960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.200651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 309/604] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:15 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/de01947b95e746f8a5d6f711d982f97e/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:15.200651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.222265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.223352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.223706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.224073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.238122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:15.252109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:15.254240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:15.256043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:15.257680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:15.257828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 296/605] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:15.258938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.356395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b452-cd7e-456b-820a-e8f75850d428 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.393180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbde0b-5cfc-4795-9e8b-7a350c728cc7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.405152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.405855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.406139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.406415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.407149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:15.407454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.408898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.422506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 310/606] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:15.423358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.444742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.445704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.446092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.446539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.449289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:15.465523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=6b6fbc800c444f2aa44f2c33c40149f0, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.466799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.486504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 297/607] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:15 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:15.487754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.570924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:15.576744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f62367-b0ca-4cbb-a815-360f80e4ddde None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.607195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6b1ca-d26e-44ba-afa6-93c889d55672 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.626538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.627220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.627506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.627785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.628831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:15.629147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.630966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.644725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 311/608] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:15.650027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.655409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.656563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.657109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.657605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.673745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.676979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:15.704189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:15.705252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:15.706151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:15.712572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 298/609] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:15.721471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.801327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:15.843672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9888d-769b-4d36-958c-8743a66e46e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.876923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389ae0-fe75-4672-ae17-a866cc14c74a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:15.885678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.886967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.887451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.887937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.902786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.904743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.917708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:15.918373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:15.918659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:15.918937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:15.919732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:15.920039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:15.921518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:15.926434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:15.927297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:15.928237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:15.928237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:15.928237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:15.928237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:15.930763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 312/610] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:15.936720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 299/611] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:15.937889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.947828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:15.972673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.107453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6428d7-d885-4db2-bb07-6706c0a87ac8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.130735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd6fed-3dd2-43dd-a3ba-f13d4e86e446 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.138075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.148882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.149571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.149841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.150120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.150862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:16.151298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.152738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.166676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 313/612] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:16.173796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.193590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.196959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.197247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.197528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/5cfd4c34ecd84abe8310817755ae62fa/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.226407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=5cfd4c34ecd84abe8310817755ae62fa, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.227951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65a229-0aa9-4f93-92f8-35c0cdf91d22 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.251919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 300/613] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:15 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/5cfd4c34ecd84abe8310817755ae62fa/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:16.286577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.316522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c95c-a108-44b5-a1eb-e8ae145b0932 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.317012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c95c-a108-44b5-a1eb-e8ae145b0932 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.317418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c95c-a108-44b5-a1eb-e8ae145b0932 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.319355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 301/614] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:16.326143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.326573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.326980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.339497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a84-b306-471e-998c-de0d8dffba2b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.346437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:16.347812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:16.382606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.383733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.384209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.384652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.393344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.395129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:16.413549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:16.414609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:16.415563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:16.418149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 314/615] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:16.427996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.441505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 302/616] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:16.468489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.570967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5bf99-6f98-4ba6-abf4-be95f8490d2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.581116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.587241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f419395-ec80-4e2d-895c-e10e55d75e8c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.604308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.612434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.613238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.613532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.613818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.614666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:16.615009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.616882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.623846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.624663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.624950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.625225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.648572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:16.652524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 315/617] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:16.657595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:16.662499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:16.664054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:16.665127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 303/618] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:16.675359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.801822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d055-d41c-47ec-8d53-a4b33d2f9645 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.858100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597681-10a1-45bc-94b9-2cc6b84473c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:16.892220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.892908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.893189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.893466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.894217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:16.894521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.895927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.898493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:16.899347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:16.899577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:16.899851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:16.905052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 304/619] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:16.916998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.918068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:16.919529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:16.932971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:16.938085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:16.939264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:16.940229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:16.940229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:16.940229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:16.940229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:16.940229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 316/620] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:16.945479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:16.995144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:17.076373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba3258-90f2-4795-9010-b7f1d8805b3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.121344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.122009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.122299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.122580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.123311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:17.123622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.125022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.150735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 317/621] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:17.157735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.181256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6796-ad70-40b7-8207-83edfa479d39 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.225988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.226790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.227076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.227359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.254503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.258676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:17.284025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:17.285154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:17.286085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:17.286085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:17.286085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:17.286085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:17.286085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 305/622] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:17.293149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.378903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd7a92-bc6a-4c12-b6fe-53bdb123c512 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.382834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:17.445068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:17.471228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65acde1-169d-46e5-a0ba-18fe904a7e5e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.482106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.482912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.483200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.483476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.494544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.496015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:17.521072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:17.523487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:17.525845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 318/623] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:17.535380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.541599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.542278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.542565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.542843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.543584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:17.543893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.545311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.579134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 306/624] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:17.584181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.660814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fe7c6-7fb4-46b8-b9e4-9ae109896dcb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.716730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.716730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.716730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.716730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.717527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:17.717527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.719002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.742719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604dba-d064-4740-978f-f78aa3abda7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.746286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 319/625] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:17.751846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.775161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:17.781618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.782602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.783327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.783438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.791518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.793529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:17.814556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:17.815782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:17.816980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:17.822058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 307/626] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:17.822508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:17.823835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:17.939553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f98a06-7583-47a7-ae66-3904cecc4c7e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:17.978744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:17.984250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:17.999538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97507e-2db6-4e59-af50-fa6afb5f92c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.004745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 308/627] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:18.007804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.044908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.114931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0879-6d02-4515-89a9-5733c951f973 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.132729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.134213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.134733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.135218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/1d1c9cf9b02a41f39f46e878747cea3f/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.151526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=1d1c9cf9b02a41f39f46e878747cea3f, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:18.152503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.153370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.153650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.154038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.154825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1392f70-3aea-4ab9-9c00-5919021532eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:18.170062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 320/628] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:17 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/1d1c9cf9b02a41f39f46e878747cea3f/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:48:18.171832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78933bea-9774-47a1-b238-e788a681283c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.172482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78933bea-9774-47a1-b238-e788a681283c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.172989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78933bea-9774-47a1-b238-e788a681283c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.180049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 321/629] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:18.181941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.182464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.182992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.202949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:18.204979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:18.234457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:18.236153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:18.284015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.310795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 322/630] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:18.354381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.356529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.411396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 309/631] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:48:18.413251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.508274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2758-6545-4e5b-92b4-cd59ff5d5e3b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.527641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aedf0-6512-4d09-b7ee-e57be9e4841b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.549508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.550395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.550703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.551091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.562886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635829fb-2020-4570-9efb-794a070d178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.563402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:18.563678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635829fb-2020-4570-9efb-794a070d178b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.563957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635829fb-2020-4570-9efb-794a070d178b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.564226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635829fb-2020-4570-9efb-794a070d178b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.566986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4af0fb01-3a7e-4d14-a357-43b4ab968bdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:18.574831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:18.575796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:18.576671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:18.577330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635829fb-2020-4570-9efb-794a070d178b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:18.577909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635829fb-2020-4570-9efb-794a070d178b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:18.580708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 323/632] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:18.582168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc2a6-c21e-4dfb-999e-eea3cc0bbae7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.582463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc2a6-c21e-4dfb-999e-eea3cc0bbae7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.582757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cc2a6-c21e-4dfb-999e-eea3cc0bbae7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.584561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 324/633] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:18.590878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.591171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.591456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-635829fb-2020-4570-9efb-794a070d178b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:18.596126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:18.598342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 310/634] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:18.600560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635829fb-2020-4570-9efb-794a070d178b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.611011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:18.612258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:18.682219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.703804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635829fb-2020-4570-9efb-794a070d178b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.704876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 325/635] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:18.706111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.740056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.741437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.741734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.742010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.742817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:18.743123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:18.744656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:18.753594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 311/636] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:18.754459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.806906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:18.836648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea6618-6cfe-4ccc-917c-dcfcbe5e72f4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.875282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33d47-cd98-4ced-94ce-4eba677c07a0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:18.883329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.884026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.884394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.884735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.885567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:18.885930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:18.887425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:18.901140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 326/637] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:18.902256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.917835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:18.918602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:18.918883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:18.919156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:18.934190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:18.945062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:18.947431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 312/638] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:18.949221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:18.965264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.031028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50a8d4-e1cd-4c1f-8472-a8de7612d904 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.068014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc862d-4e7f-43e9-a3ef-194d7000ca89 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.072347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.073170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.073454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.073727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.080192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.085636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.087073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:19.105599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:19.106510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:19.107418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:19.107418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:19.107418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:19.107418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:19.107418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 327/639] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:19.108183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.113604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.114825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.114825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.115093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.123258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.125174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5d9bd434-21ba-4949-a4f4-47f5b2ce3759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:19.140875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:19.141854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:19.142771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 313/640] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:19.144030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de70a-9d96-4313-ad23-378cb2312cb2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.144030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de70a-9d96-4313-ad23-378cb2312cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.144030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de70a-9d96-4313-ad23-378cb2312cb2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.148692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 314/641] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:19.148692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733de70a-9d96-4313-ad23-378cb2312cb2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.226970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0357b8-7315-4774-973c-621ccb9e227b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.230386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.263725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733de70a-9d96-4313-ad23-378cb2312cb2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.266139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.266804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.267097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.267380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.268130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:19.268443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.269841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.282884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 328/642] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:19.282884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.307982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.308669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.308959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.309236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.309968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:19.310281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.311707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1aaf6-0db0-44e2-abf5-ac4e5fb6538b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.325165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 315/643] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:19.326481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.326797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.327132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.344508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:19.345823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:19.374450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.396933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99df73-5f4c-4a4c-a9b2-117095600461 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.436643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 316/644] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:19.436643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.446983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.447798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.448286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.448694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.449660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:19.450092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.452273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.467346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 329/645] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:19.468470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.501588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.545491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddf249-4900-4166-b1f4-636c74b978ee None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.583568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.584341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.584761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.585047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.586481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24187b-0485-42b6-afb3-fad2800cdc99 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.596123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.597633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:19.616961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:19.617879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 317/646] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:19.618923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.628715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.629803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.630286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.630566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.637381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.639718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.641692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:19.657190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:19.658148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 330/647] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:19.663721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.737866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c94c5-e227-404b-9dff-d46ad9192349 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.766068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.775479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.776355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.776665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.776960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.783463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57700bdb-9b1f-4b42-beca-92433b2379b8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.788552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:19.797440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:19.798362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:19.799196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:19.803453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 318/648] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:19.804395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.822051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:19.822700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:19.822983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:19.823263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:19.824094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=41cfa205823a4c29abea3001bf086711) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:19.824411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:19.825902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:19.840778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 331/649] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/role_assignments?user.id=41cfa205823a4c29abea3001bf086711&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:19.840778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:19.906011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:19.951010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b37d94-ffbe-42c8-92e0-61262ed0a8b4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.960526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2aa3b-3826-40a0-a9cb-80d38d5be1b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:19.999621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.000556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.000693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.001052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.001287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.001515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.001615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.001837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.002516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.002696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.003014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.003117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.004483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.004687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.030592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 332/650] 10.222.0.61 () {56 vars in 1137 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.031605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.033432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 319/651] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.034912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.054278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.160125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009b22-9c43-4361-9e59-23b50ec0c77a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.171200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620deb4-b4e9-433e-b34a-e3d4b1be9f4a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.200034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.203757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.204768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.205183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.205924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.215012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.215921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.216357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.216878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.218257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.218719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.220234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.232392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 333/652] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.238766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.299575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.301236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.349107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.368661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32788c72-8d14-4f36-acf6-fd3d36a6bf41 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.422077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.423121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.423556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.423971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.433046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.434704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.446036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 320/653] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:20.447255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:20.454498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:20.455654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:20.456900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:20.459226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 334/654] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:20.460777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.488765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.565289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c730a-f974-44e8-8614-ea41d945d278 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.601693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.602482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.602586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfdec1-9401-4b52-8963-004a62b75cc3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.602773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.603026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.613790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.615249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.633160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:20.633377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:20.634294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:20.635210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:20.635210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:20.635210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:20.635210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:20.635210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 321/655] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:20.635750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.644811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.645605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.645891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.646167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.658078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.659544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-95c73f9b-0b1f-458c-8b85-d9e8c01bf9cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:20.677434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:20.678564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:20.679446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:20.679446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:20.679446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:20.679446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:20.683348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 335/656] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:20.684689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7363f-b968-4a69-975b-a6fd846ec342 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.684985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7363f-b968-4a69-975b-a6fd846ec342 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.685265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7363f-b968-4a69-975b-a6fd846ec342 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.688798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 336/657] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:20.688798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7363f-b968-4a69-975b-a6fd846ec342 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.731348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.750077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751451-e3ab-48e8-a976-ed080dbb9c99 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.789343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.790017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.790299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.790571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.791621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.791931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.793567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.798708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7363f-b968-4a69-975b-a6fd846ec342 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.817120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 322/658] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.817120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.840255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.840945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.841237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.841521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.842247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.842558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.843921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060b9db-33fd-4eb1-a406-6e772064ab84 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.860856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 337/659] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.860856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.860856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.860856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.876133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:20.877376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:20.915296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:20.927003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ff578-2851-4378-93bd-c1359e0175c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:20.949398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 338/660] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:20.950584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.971659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:20.973099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:20.973754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:20.974035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:20.975145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:20.975458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:20.976870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:20.989735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 323/661] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:20.991300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:20.994152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.067445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26726e-b8cb-42f7-8d3d-ef7d1b340756 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.108299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.109509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.110053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.110331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.114603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ed05b-33d4-4574-baa5-37d7de416e98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.154184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.154979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.155258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.155533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.162119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.163643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.165234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:21.179431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:21.180568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 324/662] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:21.181895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.198324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.199921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.249182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.292023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4178e-02a8-45ac-bd84-257c14036a34 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.311943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 339/663] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:21.312881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.331578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.332662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.333244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.333698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.344752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:21.354408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:21.355366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:21.356256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:21.363311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 325/664] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:21.363311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.388662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.421145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5377ad0-1973-42cd-a5df-92bd1ae717e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.455913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.456744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.456932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.457208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.464243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.465722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.469694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52d79-a15c-4460-ba81-863e30788d9e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:21.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:21.481127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:21.482017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 340/665] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:21.482799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.498729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.511821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.512488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.512818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.513095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.513933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:21.514239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.515679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.534111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 326/666] 10.222.0.61 () {56 vars in 1139 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:21.534736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.596892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c54d9-aaf4-42a0-a0d7-292809d73844 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.628396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.634965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.636016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.636289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.636567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.645413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ca1bb-6cad-4533-a073-cbbbfb825654 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.650747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.652415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:21.673653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:21.674575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:21.675455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:21.675455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:21.675455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:21.675455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:21.675455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 341/667] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:21.676286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.683006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.683680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.683961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.684409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.685192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:21.685496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.686878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.700576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 327/668] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:21.700576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.746135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.791818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706f6c2-e79d-46d6-8d42-63f1b17261ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.810299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290fb9-bc32-4b75-9d13-f5fc46bf0c1f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:21.827324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.827989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.828276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.828554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.829374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:21.829685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.831111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.843590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 342/669] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:21.844410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:21.854564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b62b48-d524-4328-8952-a201023492df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:21.855483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62b48-d524-4328-8952-a201023492df None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:21.855910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62b48-d524-4328-8952-a201023492df None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:21.856316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b62b48-d524-4328-8952-a201023492df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:21.892634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:21.957801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b62b48-d524-4328-8952-a201023492df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:21.960368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b62b48-d524-4328-8952-a201023492df None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:21.979452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5ef1f-dd7f-40f7-b79b-dfe9de3c63c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.019171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.033904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.034704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.034998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.035279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.036148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:22.036340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.038687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.057561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 343/670] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:22.057561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.093077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 328/671] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:22.094414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b62b48-d524-4328-8952-a201023492df None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.174742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632e0f-6e72-4401-b048-24bcad160a66 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.177059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.211712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b62b48-d524-4328-8952-a201023492df None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.212668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.213491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.213766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.214034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.227832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.230938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.253467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.254117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.254401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.254675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.257110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=de01947b95e746f8a5d6f711d982f97e) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:22.257424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.258908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:22.262493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:22.263482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:22.264552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:22.264552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:22.264552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:22.264552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:22.264552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 344/672] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:22.265402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.276972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 329/673] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/role_assignments?user.id=de01947b95e746f8a5d6f711d982f97e&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:22.276972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.286104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.399440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d792f4-350e-4987-95d7-6509b5c4a199 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.399697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59dce8-eab1-4968-8668-73a6ee2bef6c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.449383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.450194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.450482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.450761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.454893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.455693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.455976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.456329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.463965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.464943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.465509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.467371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-78e1ffa9-3e88-4864-a96f-40b8698c1979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:22.483924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:22.485600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:22.487213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-946d292a-6c6c-4c8a-8815-84fd372c070f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:22.487982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:22.489895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:22.492752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:22.492752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:22.492752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:22.492752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 345/674] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be18bce-98e3-403c-b51c-6d54ad6dbe6e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be18bce-98e3-403c-b51c-6d54ad6dbe6e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be18bce-98e3-403c-b51c-6d54ad6dbe6e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 346/675] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:22.495689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be18bce-98e3-403c-b51c-6d54ad6dbe6e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.496544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 330/676] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:22.496544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.496544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.496544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.512059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.517855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:22.519216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:22.583366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.599009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 331/677] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:48:22.604653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.622952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be18bce-98e3-403c-b51c-6d54ad6dbe6e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.663145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.664247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.664732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.665169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.665906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:22.666218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.667588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.696858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 347/678] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:22.698322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.714631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a341-3f62-4a36-bde9-e1a2c3d9d3f3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.756555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.757432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.757691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.757963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.759032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:22.759032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.760401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.774393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 332/679] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:22.775304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.798739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.827152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8d55d-5810-4bdf-8bf2-ada41ec59703 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.839167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:22.867347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.868019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.868307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.868585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.869439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:22.869752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.871190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.889034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 348/680] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:22.889953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:22.910958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df4f5-6041-4118-8992-038326f4d926 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:22.959082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:22.960235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:22.960525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:22.960866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:22.961539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:22.963043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:22.978419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:22.979717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:22.980547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 333/681] 10.222.0.61 () {56 vars in 1129 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:22.981949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.021135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201d992-ed0c-4ba4-a203-c5481463c37f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.058919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.059964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.060414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.060729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.063547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.070706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.072262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:23.088050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:23.089350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:23.090296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 349/682] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:23.090902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.124557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a34ed-94d1-4756-b1f9-fe036b243a58 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.139306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.164157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.165108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.165391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.165667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.173127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.174711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:23.190695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:23.191845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 334/683] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:23.192787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.204403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc61b-f437-4f2b-ab89-44cbfbb88b57 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.241849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.242509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.242799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.243080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.243923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.244233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.245700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.251705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 350/684] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:23.252509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.330547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.338349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6c87-f9f0-4d62-91c4-9ca77299d855 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.386769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.387359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.387647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.387965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.388776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.389092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.390121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9413de-2a51-4270-9f0e-6f797717ae3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.390508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.400653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 335/685] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:23.400653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.445677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.450008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.452184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.452748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.453229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.455309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.455874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.458340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.484158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 351/686] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:23.485585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.531580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2dd9-ce6f-49b7-8307-b18c67e908f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.587696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.587696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.587696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.587696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.588117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.588117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.588117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.604806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 336/687] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:23.605704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.620204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61252d03-cbff-4670-b097-9a6cb17177f6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.654047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.658818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.659588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.659875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.660154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.660983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.661875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.663625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.673174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 352/688] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:23.674144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:23.709904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:23.720226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd68e-4561-4689-9428-19300c9ded51 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.766347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.767191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.767484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.767769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.824891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b70749-ab8b-4eda-a550-1869bc6b4eed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:23.856856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.858508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.873669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:23.874339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:23.874628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:23.874938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:23.876350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=5cfd4c34ecd84abe8310817755ae62fa) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:23.876697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:23.878163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:23.891817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 353/689] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/role_assignments?user.id=5cfd4c34ecd84abe8310817755ae62fa&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:24.027997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:24.034519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 337/690] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:24.040820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:24.100461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:24.163991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ebad-bf75-4009-94e2-8b32ace7bcf6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:24.199986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:24.200846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.201136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.201418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.213406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:24.214885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:24.233852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:24.234787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:24.235651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:24.235651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:24.235651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:24.235651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:24.235651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 354/691] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:24.241245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:24.347600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c17bb-16c4-4051-a2e9-dcb4f67b0b18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:24.387360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:24.388115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.388434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.388795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.389642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:24.389981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:24.391634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:24.406312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 338/692] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:24.411457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:24.516995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:24.519550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e1cdc-baee-4732-9ffe-3ec3f12d9656 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:24.556191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:24.557156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.557440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.557716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.564943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:24.566399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:24.581788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:24.582685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:24.583569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 355/693] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:24.589142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:24.657733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:24.704828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b8ba7-0789-474c-a99a-0f3ba8b3dacb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:24.741929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:24.742602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.742887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.743166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.743962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:24.744272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:24.745820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f5032-fae7-4ef2-841f-5ce09c60f83e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:24.759214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 339/694] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:24.769973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:24.910947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3137dd-d058-4c05-9c86-fa1701ea7290 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.911245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3137dd-d058-4c05-9c86-fa1701ea7290 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.911429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3137dd-d058-4c05-9c86-fa1701ea7290 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.913352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 340/695] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:24.918760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.919052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.919332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.924534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f360-5374-44c5-b81c-c1a908b1ecaf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:24.958837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:24.964853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:24.976164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:24.976869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:24.977157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:24.977485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:24.978384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=1d1c9cf9b02a41f39f46e878747cea3f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:24.978700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:24.980445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:24.991249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 356/696] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:24 2022] GET /identity/v3/role_assignments?user.id=1d1c9cf9b02a41f39f46e878747cea3f&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:25.077473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 341/697] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:48:25.113126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:25.156754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:25.251859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddd45e-28cd-4c5d-aab0-cde73a7b5ce9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:25.290446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:25.291251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:25.291541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:25.291824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:25.303522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:25.305000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:25.322613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:25.323563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:25.324447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:25.324447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:25.324447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:25.324447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:25.324447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 357/698] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:25.330392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:25.392909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:25.453151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e8465-a992-4a5c-8de3-9d126a8f7f7c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:25.497098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:25.497765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:25.498053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:25.498331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:25.499066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:25.499377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:25.500799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9bad5-6db0-4ab4-9007-b4006a06e625 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:25.513316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 342/699] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:25.518745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:25.602433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:25.632589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aadf8-3e5b-42a4-b7b5-ec695e4fc292 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:25.671098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:25.671797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:25.672086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:25.672369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:25.673148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:25.674624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8445808-8908-4be5-88a0-1c291bf2bc2c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:25.710600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 358/700] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:48:25 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:25.912142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dad98-b407-4052-a96d-a0a609ab5855 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:25.912704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dad98-b407-4052-a96d-a0a609ab5855 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:25.912935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dad98-b407-4052-a96d-a0a609ab5855 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:25.914318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 343/701] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:25.929441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:25.929740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:25.930003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:25.949304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:25.950548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:26.038429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 359/702] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:26.062065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dad98-b407-4052-a96d-a0a609ab5855 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.174903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dad98-b407-4052-a96d-a0a609ab5855 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:26.182944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:26.214382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:26.215180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.215462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.215735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.226557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:26.234635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:26.235559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:26.236442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 344/703] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:26.242341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.324774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:26.377330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266facff-1d1b-48f5-b8ca-c2f45f00599f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:26.428357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:26.429028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.429313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.429590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.430345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:26.430652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:26.432029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850536f9-c490-4f68-9ab8-e2d1e22c3234 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:26.441523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 360/704] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:26.447529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.604924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b476-9bc2-4a0b-abc6-0c31467f9b2b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.605460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b476-9bc2-4a0b-abc6-0c31467f9b2b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.605460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b476-9bc2-4a0b-abc6-0c31467f9b2b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.607229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 361/705] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:26.612197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.612494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.612802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.618406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6f9f-b5c8-4401-ae9b-c0cec88eebf7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:26.648741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:26.650935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:26.669093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:26.670270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.670752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.671335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.686043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:26.687818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:26.707762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:26.708688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:26.709681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:26.709681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:26.709681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:26.709681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:26.711914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 345/706] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:26.717410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.757001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:26.793498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 362/707] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:48:26.816684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.860398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd55018-2892-441e-9829-50e974aa37da None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:26.895326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:26.895995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:26.896281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:26.897468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:26.898210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:26.898517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:26.899931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:26.913728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 346/708] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:26.918980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:26.953056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.044796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b5912-f006-4372-b6c1-20b3b508fb14 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.085006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.101122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.101794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.102075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.102349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.103080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.104726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5cd91-d61b-43b7-ba5a-b6f7eb13830c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.122530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6529-d96e-4783-90f8-accb2a394719 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.133043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 363/709] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:48:26 2022] POST /identity/v3/services => generated 251 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:27.141697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e1a63-100d-4124-badc-152856c8f0a7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.142044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e1a63-100d-4124-badc-152856c8f0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.142487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e1a63-100d-4124-badc-152856c8f0a7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.144261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 364/710] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:27.149913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.150198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.150478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.168812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:27.170023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:27.185953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.186779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.187062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.187392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.205842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.207265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:27.236992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:27.238031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:27.239010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:27.247817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 347/711] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:27.253471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:27.301597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 365/712] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:48:27.324167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:27.398929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82cd8-abc4-4cc3-8447-fd1bd82e1385 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.401818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.441475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.442158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.442446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.442725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.443614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:27.443927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.445579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.448919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.465589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 348/713] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:27.471583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:27.473453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8000738-d09f-4811-a56c-38a33c18b3a5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.519789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.520816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.521181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.522204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.523128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.525494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:27.540177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:27.541452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:27.542500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:27.545297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 366/714] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:27 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:27.551116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:27.606813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924612b2-0758-445f-8364-3605ba06d3ee None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.644251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.645210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.645590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.645987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.686499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec81dbc-00c3-442c-b700-67b134c7ba8d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.696270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.731038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.731997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.732464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.732943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.734375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:27.734995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.736602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.743946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 367/715] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:27.752673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:27.774263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:27.791715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.793346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99a129-e305-42fa-923d-f27cc9a8f8b0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.901041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd4175-402a-4042-bfc0-406714e2f7eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:27.969515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:27.970803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:27.971303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:27.971810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:27.973267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:27.973836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:27.976648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:27.999289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 368/716] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:28.111831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 349/717] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:27 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:28.117185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.165155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:28.190395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c076-1583-4115-b7a2-aea8ae36e7c5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.190728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c076-1583-4115-b7a2-aea8ae36e7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.190956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c076-1583-4115-b7a2-aea8ae36e7c5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.192460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 350/718] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:28.202597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.203613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.204380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.226219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:28.227783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:28.259809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cc7fb-22f2-4e7a-ac4f-f5256334b38a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:28.300570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:28.301601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.301989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.302368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.314135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 351/719] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:28.317507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:28.319178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:28.338644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:28.339733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:28.340814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:28.340814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:28.340814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:28.340814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:28.341715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.342839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 369/720] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:28.350614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.377029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:28.519724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:28.525596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a115d0-bf71-4807-8173-a53bc34c93b7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:28.538256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e02a8-2023-4819-95a2-f5b4df0773e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:28.576408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:28.577351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.578352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.578927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.580639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:28.581570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:28.583510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:28.586976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:28.587960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.588384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.588850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.612092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 370/721] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:28.614015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:28.621418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:28.632682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:28.633879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:28.634939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:28.641970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 352/722] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:28.647536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.800426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:28.803018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b218-7b04-419b-ae6c-3c70b978a422 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:28.836185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4f798-df7b-4994-aa3e-0eed7df89a84 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:28.859119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:28.860259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.860777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.861374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.869235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:28.870871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:28.877477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:28.878403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.878694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.878977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.879736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:28.880047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:28.882422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1da77b74-54b6-4543-9889-fbe65dd225b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:28.889274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:28.890443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:28.891567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:28.893392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 371/723] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:28.894761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 353/724] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:28.895012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e22fe-6e6a-48ec-826c-4908490cc332 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:28.895366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e22fe-6e6a-48ec-826c-4908490cc332 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:28.895723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e22fe-6e6a-48ec-826c-4908490cc332 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:28.904541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 372/725] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:28.909630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e22fe-6e6a-48ec-826c-4908490cc332 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.909770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:28.919315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.033156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e22fe-6e6a-48ec-826c-4908490cc332 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.054272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1e26-c457-413c-84bf-77494cab98c3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.078347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.079736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.080451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.080903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.082111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:29.083014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.085379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e2a7b-6769-4223-bdef-b797f85eb196 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.093833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.094699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.095055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.095410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.101365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 373/726] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:29.103419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.104153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.104607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.107506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.109271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.112388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.127747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:29.129203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:29.130769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:29.131767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:29.132934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:29.132934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:29.132934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:29.132934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:29.132934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 354/727] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:29.134027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.208785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.227829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 374/728] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:29.227829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.271046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25198051-55f7-404f-b09b-b4b4973812b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.318455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.319619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.319866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.320115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.320986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:29.321297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.322746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.329564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 355/729] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:29.330508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.353582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c87b-7980-4771-9caa-7ba16a469eed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.394154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.395226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.395732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.396215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.398329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:29.398919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.400239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.401212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.416385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 375/730] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:29.425821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.452684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e490-c993-4348-90b1-ac2c7364dcff None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.487168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.487979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.488260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.488595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.501850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:29.510847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:29.511835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:29.512705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:29.516765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 356/731] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:29.518549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.613258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.643212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f9442-cbd9-4250-893b-63d6d825ac68 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.668201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:29.688123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67de1a-bb84-489a-a735-c02cb02f65b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.701729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.702625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.703060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.703482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.704406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:29.704883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.706561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.727063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 357/732] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:29.733894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.761622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.762704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.763133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.763552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:29.779538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:29.782209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:29.802135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:29.804881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:29.806082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:29.808403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 376/733] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:29.813419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:29.942509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723844a8-4217-4969-b802-2632edafd2d3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:29.988152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:29.989532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:29.990019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:29.991707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.009831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669f6e-5ccf-4c37-b12e-307f3e5ab0fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.018223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.019722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.041060 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.062868 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.063957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.064455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.064974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.066017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:30.066545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.068300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:30.069734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:30.070597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:30.071488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:30.071488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:30.071488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:30.071488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.071488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 358/734] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:30.077971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.102304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 377/735] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:30.112740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.157994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.231826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490bc5b-e241-41a9-b656-0c5f4501ee21 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.270534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.271238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.271804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.271804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.272845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:30.273156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.274609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa2b21-4e92-4254-85ae-f8cc2d20de86 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.275042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747601c8-69a2-47c8-94ab-711fd5a46566 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.289753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 359/736] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:30.291177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daab95-f1a5-4887-9a15-3159e2b8470d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.291466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daab95-f1a5-4887-9a15-3159e2b8470d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.291745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daab95-f1a5-4887-9a15-3159e2b8470d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.297032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 360/737] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:30.297032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daab95-f1a5-4887-9a15-3159e2b8470d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.332762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.333418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.333703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.334178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.393930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.414141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.426655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daab95-f1a5-4887-9a15-3159e2b8470d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.435396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.437308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e0929-c58b-483a-b775-c4c7262d4a74 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.466460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.467519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.468205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.468488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.478688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.480982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-91878f0f-8f4e-408b-bead-1de7e694d95e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:30.498972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:30.500113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:30.501302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.504874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 361/738] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:30.506294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.506587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.506873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.524220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:30.525438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:30.547122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 378/739] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:30.548576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1ce9b-d43f-4d8d-b86a-d713a249e8c7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.548999 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1ce9b-d43f-4d8d-b86a-d713a249e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.549405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1ce9b-d43f-4d8d-b86a-d713a249e8c7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.553415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 379/740] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:30.553415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1ce9b-d43f-4d8d-b86a-d713a249e8c7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.604059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 362/741] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:30.605258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.643760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.672810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1ce9b-d43f-4d8d-b86a-d713a249e8c7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.709855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.711989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.712922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.713274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.713647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.714563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:30.714935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.716418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a6143-9c5b-4752-a56a-ca3f1f6ff5a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.729480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb341-4cd7-47a2-b079-3bfb74725951 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.737873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 380/742] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:30.739354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.739744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.740788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.758420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:30.759547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:30.774103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.774890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.775177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.775453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.785996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.788300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:30.805691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:30.806601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:30.807491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:30.807491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:30.807491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:30.807491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.808917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 363/743] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:30.810661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.834975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 381/744] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:48:30.835825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:30.918268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.930327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e353d-004e-483e-a971-03f99770a381 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.962611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e72514-2781-4dc0-ac45-36decc17986c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:30.964119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:30.964964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:30.965250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:30.965600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:30.966273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:30.967660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:30.970248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:30.981924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:30.982865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:30.983687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 364/745] 10.222.0.61 () {56 vars in 1129 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:30.984533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.012609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.013219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.013507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.013785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.091199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.092692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.101222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc56fa-bddb-427f-8385-dad3db2e8e22 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.137721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.138680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.139110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.139386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.140109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:31.140467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.141819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.154408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 365/746] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:31.155212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.161586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.214490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 382/747] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:48:31.214490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.222310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.268071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575be0e-d079-4ece-82e1-2ddf439b25e3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.311590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.312255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.312533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.312844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.313636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.314938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.326245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a9ef-6bdc-41fe-9ec6-c32c032ad572 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.329562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 366/748] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:48:31 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:31.330276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.434328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.436145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.437203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.437935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.440324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:31.441161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.445050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.459729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 383/749] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:31.459729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.514692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.519768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8e75a-3f20-464f-93bf-d1fafc0c1feb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.546574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.567363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.568306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.568609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.568915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.581636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.583124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:31.604863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:31.605826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:31.606732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:31.608690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 367/750] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:31.609731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.632405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd9e3-92d7-4c09-9988-e29225bdda3e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.680528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.681373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.681658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.681934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.697571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.699661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.725573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bfe85-b872-4df7-916f-fec8c39c0eb4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:31.726177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:31.727050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:31.727941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:31.727941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:31.727941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:31.727941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:31.732135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 384/751] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:31.734151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.769140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.770299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.770801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.771357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.772370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:31.772936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.774602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.781712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 368/752] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:31.782784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.838818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.839340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:31.869000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a8826-9bbc-4940-bd6a-909c2edcbf5a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.906468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc7d8a-5c9f-43ab-8ebf-4078e05ebbe6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:31.911586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.912281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.912567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.912885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.913840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:31.914153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.915716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.934457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 385/753] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:31.934457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:31.946047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:31.946915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.947264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.947545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.948301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:31.948643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:31.950133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f5828-f3e0-4fd5-909f-b97ecae6aeaa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:31.962941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 369/754] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:31.964223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33392-8638-4385-96f9-6ec6b10fcade None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:31.964515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33392-8638-4385-96f9-6ec6b10fcade None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:31.964930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33392-8638-4385-96f9-6ec6b10fcade None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:31.966157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 370/755] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:31.966855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d33392-8638-4385-96f9-6ec6b10fcade None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.048962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c8df-1a6d-4884-b7fc-03c6c8751b4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.084413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d33392-8638-4385-96f9-6ec6b10fcade None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.087553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.090677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.091382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.091742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.092103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.093017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] RBAC: Adding query filter params (interface=public, service_id=b7c1211a1ffc4f2799909f3a477fc030, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:32.093402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.094874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.096811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.104408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 386/756] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/endpoints?service_id=b7c1211a1ffc4f2799909f3a477fc030&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.105484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.129886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.130298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.130586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.131461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.132296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=1d1c9cf9b02a41f39f46e878747cea3f) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:32.132611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.134294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9403888-010f-4807-87d6-0058f6baeaf1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.140182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 371/757] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:31 2022] GET /identity/v3/role_assignments?user.id=1d1c9cf9b02a41f39f46e878747cea3f&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.141929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.142224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.142503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.167228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:32.168247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:32.261151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 372/758] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:32.262607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.370586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced92e1-c954-47e4-9576-f385f921ff76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.398380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.403819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142a38a-d110-4322-8e11-ab259d08b9aa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.414021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.414819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.415104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.415382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.425036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.426486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.443733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:32.444591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:32.445586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:32.446494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:32.447847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.448394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.448894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.449808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.450713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 387/759] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:32.451595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.452060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.458933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 373/760] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.460309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.567364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.587694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62e49-5e44-42dc-963c-62a9b9caa759 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.651182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.652394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.653818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.654972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.656195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:32.658321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58885d-1445-40d9-8683-7f9c8f7a3e34 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.658921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.660668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.686179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 374/761] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.693768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.731032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.744700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.745493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.745760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.746048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.759548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:32.767193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:32.768355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:32.770132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 388/762] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:32.776444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:32.832460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2284d-bb89-4f89-a303-d13572b81a8c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.887572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.888284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.888575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.888898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.889743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=d04bed939eac45d2907980fea2c644d8) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:32.890060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.891509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.903249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 375/763] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/role_assignments?user.id=d04bed939eac45d2907980fea2c644d8&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.913239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d15656-a9d6-4ca5-a2a9-443765d83d22 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:32.946055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:32.971451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198db29-341d-4005-83a3-9fce216032df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:32.972181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198db29-341d-4005-83a3-9fce216032df None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:32.972479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198db29-341d-4005-83a3-9fce216032df None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:32.972785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198db29-341d-4005-83a3-9fce216032df None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:32.973556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198db29-341d-4005-83a3-9fce216032df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:32.973869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198db29-341d-4005-83a3-9fce216032df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:32.975483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198db29-341d-4005-83a3-9fce216032df None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:32.986146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 389/764] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:32.992395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:33.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:33.133565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded11d-7a8d-418e-a90b-a7cb216301d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:33.170988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:33.171771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.172110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.172395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.186475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:33.188025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:33.208908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:33.210200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:33.211082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:33.211082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:33.211082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:33.211082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:33.211082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 376/765] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:33.217133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198db29-341d-4005-83a3-9fce216032df None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:33.355156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198db29-341d-4005-83a3-9fce216032df None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:33.406205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:33.406937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.407272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.407572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.408381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:33.408750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:33.410216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:33.430990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 390/766] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:33.436473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:33.554424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567e30-b0c1-439f-bbb4-218d64f7c234 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:33.591223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:33.592095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.592380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.592702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.601457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:33.603017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:33.605174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d4ad7efb-7ca7-4de6-9607-ddccbdfcf17c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:33.618649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:33.619831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:33.620965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:33.626017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 377/767] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:33.632277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:33.699215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:33.713613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7c33-afd8-4666-9138-bacfd9ac01f8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.713897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7c33-afd8-4666-9138-bacfd9ac01f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.714182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7c33-afd8-4666-9138-bacfd9ac01f8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.715555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 378/768] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:33.730605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.730884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.731164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.753489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:33.754696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:33.800550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097152e-c2ac-485d-8a88-83d2744d3c72 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:33.864726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:33.865648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:33.865923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:33.866205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:33.867047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:33.867358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:33.869581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:33.873648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 379/769] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:33.889299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 391/770] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:33.894884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:33.912997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.081193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27137-085d-40e4-a5eb-efb4905127cd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.092057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.096809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20559-b34d-46bc-8242-7a66461120d5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.133038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.146570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.148294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.148585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.148886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.160885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.161694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.162524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.162804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.163660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.165057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.169847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.171331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:34.190186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:34.191217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:34.192092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:34.192871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:34.194214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:34.195293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:34.195293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:34.195293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:34.195293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:34.195519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 392/771] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:33 2022] GET /identity/v3/services/network => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:34.195588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 380/772] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:34.198797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.199085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.335883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94742f5-8f39-42ec-aa56-d12f2e399cad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.339752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1b6-7da5-4b1d-aeec-c2f0d0f4b0da None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.378951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.380562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.381673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.382163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.383121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:34.383635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.385223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.400275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 393/773] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:34.408276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.420076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.420800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.421092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.421375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.422224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:34.422535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.423928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.436245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.439320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 381/774] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:34.445896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.527769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.588189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b321-6da4-4ff7-9ac1-ba682b74815f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.626796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066153b7-b0a0-43bc-9138-9763e7bdd871 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.629074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.629973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.630481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.631547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.632461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:34.633014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.634600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.640509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 382/775] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:34.648055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.687326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.688490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.689182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.689687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.700210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.702676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:34.721082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:34.722039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:34.722907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:34.726158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 394/776] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:34.736685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.833560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e4dd1-1155-4c94-8f34-104754aee416 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.844799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.871764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.872427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.872734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.873138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.873941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] RBAC: Adding query filter params (interface=public, service_id=6ada9d6a91e54777848be8582a3fae99, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:34.874250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.875636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.889160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 383/777] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/endpoints?service_id=6ada9d6a91e54777848be8582a3fae99&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:34.890546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:34.926174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:34.947813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596daac0-73bc-4d78-8959-2528c317b8bc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:34.985282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:34.986051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:34.986343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:34.986651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:34.987510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:34.987824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:34.989397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea39e7-fcb0-424c-a935-245e4ee0b2e3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:34.992967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdfc6b-bba2-4eb8-bca2-d4cdcb5903c1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.008611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 395/778] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:35.010137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d85dd-f2d8-4a67-9166-b1d677948698 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.010438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d85dd-f2d8-4a67-9166-b1d677948698 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.010812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d85dd-f2d8-4a67-9166-b1d677948698 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.012337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 396/779] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:35.013052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d85dd-f2d8-4a67-9166-b1d677948698 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.027721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.028387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.028695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.028974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.029657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.031319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6432e-6b1e-4b54-b194-2bd2a7df57eb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:35.044716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 384/780] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:34 2022] GET /identity/v3/services => generated 772 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:35.046258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67905b3-f901-4a3a-9860-f60b17e73f51 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.046550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67905b3-f901-4a3a-9860-f60b17e73f51 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.046827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67905b3-f901-4a3a-9860-f60b17e73f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.089759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e67905b3-f901-4a3a-9860-f60b17e73f51 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:35.091079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e67905b3-f901-4a3a-9860-f60b17e73f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:35.148819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d85dd-f2d8-4a67-9166-b1d677948698 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.164166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:35.197425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.198323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.198603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.198917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.201881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=6b6fbc800c444f2aa44f2c33c40149f0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.203316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:35.223038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 397/781] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:35 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:35.249590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 385/782] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:35.287631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.293826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:35.418354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29858ba-5879-4ea3-9a0c-8da84fdba6b8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.425740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c304-5a3e-4a2d-b98f-e889180a951a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.426483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c304-5a3e-4a2d-b98f-e889180a951a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.426977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c304-5a3e-4a2d-b98f-e889180a951a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.430624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 386/783] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:35.436930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.437225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.437505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.462401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.462610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:35.463206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.463490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.463762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.463938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:35.477902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:35.486080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:35.487021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:35.487872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 398/784] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:35.493195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.551774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 387/785] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:35.569086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:35.577552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.616901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84bf5a-a26b-441b-863c-e52ae83c40f0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.664570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.667599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.669441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.670626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.671779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:35.673785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.676227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:35.694890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 399/786] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:35.698373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:35.701289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.709727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b379a-7491-4749-9d60-3a34d3306f82 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.784958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.786159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.787448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.789389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.790274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.793688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:35.817635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:35.819041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:48:35.819869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:35.825083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 388/787] 10.222.0.61 () {56 vars in 1129 bytes} [Thu May 19 21:48:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:35.832093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:35.945658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f311bb3-b037-464b-a251-568dae29eea1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.950468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5985c-29f2-4514-8ace-527a1d23e67a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:35.970668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:35.983292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.983845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.984129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.984405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.989390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:35.991098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:35.991746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:35.992359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:35.993309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:35.993845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.995548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2433ce-cdf5-4d18-81b4-4738799364c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:35.996007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:35.997456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.001627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 389/788] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:36.003173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdc9b-3efe-489a-a3da-8df8096e3086 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.003674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdc9b-3efe-489a-a3da-8df8096e3086 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.004193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdc9b-3efe-489a-a3da-8df8096e3086 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.006658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 390/789] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:36.007683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdc9b-3efe-489a-a3da-8df8096e3086 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-57de123a-3445-4586-99cf-73044fb2e522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:36.023240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:36.024202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:36.025132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:36.025132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:36.025132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:36.025132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:36.025132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 400/790] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:36.026018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.026300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.026577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.040186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.048049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:36.049842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:36.126749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 401/791] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:36.127928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.163954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdc9b-3efe-489a-a3da-8df8096e3086 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.201728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.202585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.202886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.203173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.203902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:36.204214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.205827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.212728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 391/792] 10.222.0.61 () {56 vars in 1124 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:36.213902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.271180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74c80-ca69-46b9-a787-3fc15a7ab60c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.281793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.317418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.318127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.318474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.318754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.319482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:36.319792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.321186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.321775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.336553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 402/793] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:36.337384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.338145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d183146-d458-467d-9ed5-cf69c26d4c78 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.376646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.377827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.378356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.378978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.390565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:36.399283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:36.400193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:36.401055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:36.408470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 392/794] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:36.409558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.456252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34480f78-83a4-4f56-aa00-b4d67893f4cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.498296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.499358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.499845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.500416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.501550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.503145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.531981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 403/795] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:48:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:36.533297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.551717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ec56e-20e9-4407-b699-c5988d585aaf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.569202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.634154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.634977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.635271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.635556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.649758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.653369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.659425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9b6e-9971-4b41-ac4b-5fe4e5fd4533 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:36.680214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:36.681255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 393/796] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:36.691267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.701992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.719681 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.720920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.721522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.722037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.723199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:36.723876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.726636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.737086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 404/797] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:36.744383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.854698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:36.874367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173910e9-82a2-4610-b5ea-c28fb64c5620 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.878431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca30-b25f-4a04-880a-26255b14a1ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:36.911137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.913591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.914191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.914717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.916443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:36.917333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:36.917695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:36.918050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:36.919072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:36.919451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.921025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.929331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:36.931018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:36.941314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 394/798] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:36.949592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:36.950393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:36.951461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:36.952568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:36.952568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:36.952568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:36.952568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:36.954471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 405/799] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:36.960226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.045657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.082220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7805a-4344-4d5d-a036-8b4d3e7ee4f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.093048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ef726-6740-49ea-b1a5-fdc0f5f3dbb0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.121016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.125919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.130086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.143141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 406/800] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:37.149794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.155332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.156138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.156420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.156709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.175862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.177406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:37.220914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:37.221906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:37.222826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:37.222826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:37.222826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:37.222826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:37.222826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 395/801] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:37.228445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.341956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062affc-2ac4-48ad-a26b-539ce2229b22 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.404434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118b30-ff52-49be-b15e-4f5105b1a529 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.414853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.415912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.416346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.416797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.428030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.429709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.434561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:37.453295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:37.454281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:37.455194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 407/802] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:37.464833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.497405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.498094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.498382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.498652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.499397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:37.499706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.501132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.527298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 396/803] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:37.533316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.554906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.647139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cff34-eaf5-4bfb-9edc-377b8615ff40 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.659139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dec38-7579-4ba7-9d38-ba5cf79911c9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.685644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.686816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.687314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.687831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.695866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.697639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:37.712653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:37.713591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:37.714488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:37.716957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 397/804] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:37.722977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.739325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.740168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.740574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.740878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.741902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:37.742272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.743916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6725-6ec6-4560-9379-ad11aa3edfaf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.769228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 408/805] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:37.770647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8006d-fb02-46a3-b3e6-60affd175701 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.770990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8006d-fb02-46a3-b3e6-60affd175701 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.771368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8006d-fb02-46a3-b3e6-60affd175701 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.777219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 409/806] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:37.777876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.778754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8006d-fb02-46a3-b3e6-60affd175701 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:37.852057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a89341-b747-4d1b-9f6d-c4a5493ab556 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.889852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.890520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.890809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.891090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.891889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:37.892000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8006d-fb02-46a3-b3e6-60affd175701 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:37.892201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.893695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a80d3a-9ef3-4170-8fde-6f6825223dc9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.906936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 398/807] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:37.908000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.908293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.908574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.912901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:37.925607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:37.926810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:37.935251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:37.936038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.936320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.936594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.947566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:37.949038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cd8d5dd7-24c3-495a-824e-416e038b18a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:37.968177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:37.969146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:37.970026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:37.970026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:37.970026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:37.970026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:37.970026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 410/808] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:37.971027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07986f7e-1f46-4079-8029-821011314dad None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:37.971316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07986f7e-1f46-4079-8029-821011314dad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:37.971596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07986f7e-1f46-4079-8029-821011314dad None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:37.976760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 411/809] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:37.976760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07986f7e-1f46-4079-8029-821011314dad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.005440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 399/810] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:38.006783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.048043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.111073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07986f7e-1f46-4079-8029-821011314dad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.130793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649dec-21fa-4858-a2df-2a05f0cf5c5c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.146722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.147768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.148049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.148319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/1d1c9cf9b02a41f39f46e878747cea3f/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.150470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=1d1c9cf9b02a41f39f46e878747cea3f, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.151890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62d9e7-3bda-4304-8ee2-f1d7f8955eb2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.167164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 412/811] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:37 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/1d1c9cf9b02a41f39f46e878747cea3f/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:38.167464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.168134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.168416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.168720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.169602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:38.169902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.170014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.170487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.170986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.171325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.185036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 400/812] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:38.185973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.197193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:38.199598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:38.236371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.297095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.315928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722052a-5eb0-49fb-9891-91e112cc0b85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.320931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 413/813] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:38.322203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.381370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.381606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.381979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.382149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.393556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:38.401278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:38.402215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:38.403098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 401/814] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:38.403787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.444177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd12a-40f7-4ec6-a97c-9dcd7ee60621 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.481326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.482057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.482337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.482609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.489589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.491733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:38.506109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:38.507027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:38.507917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:38.511960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 414/815] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:38.512897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.532588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.553821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c75b-1e2a-4db0-ba1d-c61b7b6d965e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.601076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.602556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.603544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.604315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.605403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.607624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:38.622449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:38.623602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:48:38.624435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:38.625478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 402/816] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:38.626289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.628376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac50941-56e2-40c6-aa50-73ff1904971a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.641489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.679430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.680356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.681087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.681534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.682829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:38.683292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.684849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.703177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 415/817] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:38.703177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.750999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff00dcf-05e4-48a4-a476-be98be5b532f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.787811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.789066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.789633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.790121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.790595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.792579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:38.793342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.795773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.810123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 403/818] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:38.811266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.819099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f849fe-b7a0-4ed4-bfe1-e9b40434195f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.866490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:38.867169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:38.867574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:38.868307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:38.869075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:38.869822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:38.871472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:38.878476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 416/819] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:38.879255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:38.906367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:38.949136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50594f05-e689-4437-aa9c-aac7c112205c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:38.998949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.000216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.000519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.000825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.013759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.015395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.026465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d3fc8-bdcf-462b-88cb-2fe791cc7147 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:39.038832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:39.040018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:39.041160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:39.041160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:39.041160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:39.041160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:39.042599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 404/820] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:39.043549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.074341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.075490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.075949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.076374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/d04bed939eac45d2907980fea2c644d8/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.093600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:39.097093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=d04bed939eac45d2907980fea2c644d8, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.100596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.128862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 417/821] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:38 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/d04bed939eac45d2907980fea2c644d8/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:39.131721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.246344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451692-c139-4c17-bc83-0357105bbc7b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.299524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:39.316461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8777c-6352-49d3-b604-a83b5e92b9fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.332821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.334198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.334605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.334975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.335796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:39.341071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.343129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.349960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 405/822] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:39.357285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.379203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.379858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.380141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.380452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.381260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:39.381565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.382925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da0933-d2d8-4a89-8ce9-ff5608a73c7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.398140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 418/823] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:39.399298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df12f5-fc00-427c-90ed-f312c7b3217c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.399584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df12f5-fc00-427c-90ed-f312c7b3217c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.399862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df12f5-fc00-427c-90ed-f312c7b3217c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.401173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 419/824] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:39.403912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df12f5-fc00-427c-90ed-f312c7b3217c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.469997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:39.487513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410577c-a71f-47c6-b4cc-2dd5733a9925 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.527942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df12f5-fc00-427c-90ed-f312c7b3217c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.533751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.534632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.535080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.536130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.537193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.539635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f386a03-05df-4a1a-afef-1376e9e48a4f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.565877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.566602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.566889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.567170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.567757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 406/825] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:48:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:39.570269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.571467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.571962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.576560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.578185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.592194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:39.593743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:39.594068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:39.595172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:39.596322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 420/826] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:39.601699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.644490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:39.713308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 407/827] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:39.750103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.821840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdec3e4-7539-451b-b1f6-35ef9f8725ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.842431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:39.874502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.875182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.876242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.876674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.877597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:39.877914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.879877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.906187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 421/828] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:39.907740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd955c0-18e0-4429-87a2-c765ffd2c1e1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:39.912592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:39.956591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:39.957427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:39.957955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:39.958249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:39.960095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:39.961565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:39.976633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:39.977832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:39.979032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 408/829] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:39.983820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.023048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.066662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1b629-61b4-41b2-a551-fa8b328ba409 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.115442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.117413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.118118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.118618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.121804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1a9c2-a38d-478a-8396-3d88d1d94d2d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.163366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.173720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.174828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.175547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.175999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.176911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:40.178125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.180274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.187326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 409/830] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:40.193024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.198623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.199620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.307436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 422/831] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:48:40.312547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.317016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.326695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2477b5-3f40-4e40-ab0c-394964f96809 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.365984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.367174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.367786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.368168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.369022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:40.369431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.370911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.376658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 410/832] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:40.395670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.432769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56ebbe-d2b1-436b-8f2e-dba3075cde1b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.478147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.479212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.479484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.479753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.492420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.493920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0c4312dd-9bf0-44af-9d18-04c375f960c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:40.514080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:40.515053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:40.515956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:40.515956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:40.515956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:40.515956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:40.515956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 423/833] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:40.523004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.632804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b256f2-7ffa-4386-9130-6f8ca287f02d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.633458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b256f2-7ffa-4386-9130-6f8ca287f02d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.633751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b256f2-7ffa-4386-9130-6f8ca287f02d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.635129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 424/834] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:40.642056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.642355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.642640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.643089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2387b-38f8-4416-abd1-a39c3a7d7610 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.675761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:40.678452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:40.682482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.683154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.683442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.683726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.684454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:40.684797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.686191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.701038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 411/835] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:40.705868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.732566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.765678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 425/836] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:40.789442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.838378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9f735-8ef6-4192-a3ba-b4782c26950b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.847306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:40.882571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.883437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.883658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.883940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.892331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:40.894348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:40.903441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3ad3b-03c0-48a8-9734-48e864d30fac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:40.911138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:40.913019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:40.914512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 412/837] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:40.918565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:40.954394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:40.955605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:40.955942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:40.956271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:40.970829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:40.981986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:40.982952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:40.983947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 426/838] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:40.988796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.024841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.051950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d5ad-9e6e-4760-9360-7e009ec20d2c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.089221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.090131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.090597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.091034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.092162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:41.092650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.094256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.109134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 413/839] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:41.115407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.122678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17f3e0-4ad7-4ef8-8d31-f0f6eb90fcec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.136988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.161428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.162985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.163280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.163613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.164376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:41.164777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.166144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.177008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 427/840] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:41.183640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.247048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904517e-7ffb-4015-91e5-9d87eea2a953 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.285615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.286380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.286744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.287033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.288091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:41.288297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.289819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.297095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 414/841] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:41.310599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa90bf-1031-4b43-8282-7f09f20d53b7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.351212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.352336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.352828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.353261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.368132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.369884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:41.389888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:41.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:41.392193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:41.392193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:41.392193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:41.392193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:41.394236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 428/842] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:41.400177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.404920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.517579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac638b86-e3ec-4f17-ad35-a09428619374 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.553257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.553930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.554218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.554498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.555232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:41.555537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.556963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa64c1-b50c-45cf-9368-b1649701250a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.570370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 415/843] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:41.575974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.608854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.658546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016029c3-75ab-4ab1-a4cf-c0907488a0da None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.658829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016029c3-75ab-4ab1-a4cf-c0907488a0da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.659034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016029c3-75ab-4ab1-a4cf-c0907488a0da None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.660697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 416/844] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:41.666781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.667322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.667822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.685758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:41.687422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:41.688902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5fe-000f-4dc5-ad10-1f4754edede2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.725569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.726504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.726901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.727075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.735683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.737229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:41.751964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:41.752965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:41.753918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 429/845] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:41.755758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.759145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.769664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 417/846] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:41.795064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.848915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:41.865798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44aa0-ac04-492b-9b98-63e7bdb50e65 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.901175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.901834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.902120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.902396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.903348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:41.903665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:41.905125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:41.909544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21f971-839e-4c1d-b9a7-407b3aef0174 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:41.923802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 430/847] 10.222.0.61 () {56 vars in 1135 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:41.929920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:41.947048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:41.948209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:41.948790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:41.949507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:41.961162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:41.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:41.970526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:48:41.971354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:41.979201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 418/848] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:41.980600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.035231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6859d-441a-4a67-bf51-fd5980706bb9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.050701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.075494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.076444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.076905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.077405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.094526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.107836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee9574-84a0-4854-b0f0-67880b74a99e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.144224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.144973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.145285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.145560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.147380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:42.147694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.149172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.163085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 419/849] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:42.164064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.174667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.176647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.274201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683320f0-f393-4bd4-9806-fe3c63ecc224 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.288347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.299530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 431/850] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:48:42.303401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.310827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.311627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.311909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.312183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.326456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.328655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:42.350963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:42.351891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:42.352814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:42.352814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:42.352814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:42.352814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:42.356697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 420/851] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:42.358026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.360594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.429127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be7aeb-48e2-48d8-979b-0641a4305a12 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.472134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.473208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.473830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.474951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.480171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec946f-f212-4a76-a3c2-4105f4584c98 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.488037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.490459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:42.514669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:42.515807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:42.516934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:42.516934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:42.516934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:42.516934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:42.519115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 432/852] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:42.525062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.531021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.531444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.531735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.532014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.532793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:42.533101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.534494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.554515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 421/853] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:42.555394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.590236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.648954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.656254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dee3-4be9-4905-8413-d685253c3c3d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.667706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5626-6a65-4d11-a9f6-2b6331185655 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.697036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.697655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.697942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.698224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.698956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:42.699264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.700803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.706086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.707166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.707644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.708111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.716662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.718464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 433/854] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:42.718955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.719411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:42.737029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:42.738006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:42.738924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:42.746472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 422/855] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:42.747599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.832016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df151-1dec-4917-aa55-9b8c74ade7f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.842573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.859844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa45f-949f-4e8f-943e-8159fbb32264 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:42.868090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.868560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.868879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.869163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.876837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.878471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.896297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:42.896991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d1e41ee8-0b7c-44f3-9db4-03d22dd8ec5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:42.897194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:42.898049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:42.898919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:42.899341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.899341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.899341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:42.899522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 434/856] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:42.900531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f2f5-cd16-4c50-b8d0-eae879ab9f70 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.900531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f2f5-cd16-4c50-b8d0-eae879ab9f70 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.900531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f2f5-cd16-4c50-b8d0-eae879ab9f70 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.901875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 435/857] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:42.903192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:42.904691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ba28-f62a-4497-8f79-84dcb5a23ba4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:42.905673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f2f5-cd16-4c50-b8d0-eae879ab9f70 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:42.923700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 423/858] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:42.925079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:42.925369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:42.925650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:42.928709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:42.946706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:42.948014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:43.024571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f2f5-cd16-4c50-b8d0-eae879ab9f70 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.027662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 424/859] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:43.028036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.077140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.078379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.078868 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.079325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.080285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:43.080872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.082330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.094394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 436/860] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:43.095199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.117511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.145026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedfcbe-372d-4f73-836f-fd44b1258654 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.181228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.182036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.182199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.182370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.182607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.208301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e41a18-dbae-45c5-8323-fbcb37fe803d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.251138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.251808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.252252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.252546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.253811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.255948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.256075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.257547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.269605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 437/861] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:48:43 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:43.277692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.357503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.401932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 425/862] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:48:43.407123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.438041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1136756-a9aa-4177-8eea-1f6f45fdc1a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.481067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.488461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.489967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.490234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.490766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.491962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=1d1c9cf9b02a41f39f46e878747cea3f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:43.492513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.494127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.501005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 438/863] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/role_assignments?user.id=1d1c9cf9b02a41f39f46e878747cea3f&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:43.551433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0ce3-840e-41a1-9b78-f184c67262ba None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.599831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.601088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.601582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.602085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.615431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.617412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-283ba02f-0518-4af4-8ba6-197f37937375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:43.637883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:43.638869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:43.639746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:43.639746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:43.639746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:43.639746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:43.642219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 426/864] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:43.648254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.652716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e1f1-7be7-4d00-a089-e4e390907d7c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.653015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e1f1-7be7-4d00-a089-e4e390907d7c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.653228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377e1f1-7be7-4d00-a089-e4e390907d7c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.656058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 427/865] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:43.660895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.661186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.662000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.680909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:43.682148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:43.706607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.774099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 428/866] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:43.789502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef95f-9cac-4683-a22d-412b7edda428 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.796210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.824639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.825324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.825590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.825868 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.826594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:43.826896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.828256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.842616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 439/867] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:43.848083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:43.923996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:43.925618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648d124-028b-4808-970a-4e5eb0cf81f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.965553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:43.966621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:43.967073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:43.967617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:43.968551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:43.970137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:43.971433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85062-1422-413a-953e-6d69d93cae6a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:43.984606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:43.985784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:43.986823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:43.989580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 429/868] 10.222.0.61 () {56 vars in 1132 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:44.000673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:44.012191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:44.013138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:44.013517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:44.013893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:44.026246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:44.028441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:44.033618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:44.049633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:44.050769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:44.051881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:44.054349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 440/869] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:44.059551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:44.136461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43c47-b623-4a6a-b64c-df691869e4db None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:44.173427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932c908-9d21-44a1-b7a2-88bfee70fed3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:44.187891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:44.189412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:44.190174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:44.190658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:44.191439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:44.193727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:44.196850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:44.198040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:44.210146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 430/870] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:44.212089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:44.221348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:44.222016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:44.222306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:44.222585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:44.223385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:44.223698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:44.225545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:44.244042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 441/871] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:44.244963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:44.315762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:44.341635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1f04-c7e1-4be6-8097-164c61793672 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:44.366903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f917-0f47-4d6c-9cba-b39325d2d8b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:44.389788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:44.390980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:44.391445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:44.392096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:44.393459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:44.394156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:44.396467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0de0b-604c-4901-aa1b-817faf6eef07 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:44.408740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 431/872] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:44.409518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:44.410202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:44.410491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:44.410773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:44.412163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=d04bed939eac45d2907980fea2c644d8) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:44.412486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:44.414315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5154-84b3-4d34-a21f-93397f8c1ca6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:44.421616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 442/873] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:44 2022] GET /identity/v3/role_assignments?user.id=d04bed939eac45d2907980fea2c644d8&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:45.627223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593ae3-7b42-476b-b8a7-2b296c3e47e8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:45.627794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593ae3-7b42-476b-b8a7-2b296c3e47e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:45.627794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd593ae3-7b42-476b-b8a7-2b296c3e47e8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:45.629446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 432/874] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:45.635391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097262a-e809-447b-93e1-644ff417ec2c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:45.636208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097262a-e809-447b-93e1-644ff417ec2c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:45.636666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097262a-e809-447b-93e1-644ff417ec2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:45.656097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2097262a-e809-447b-93e1-644ff417ec2c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:45.657589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2097262a-e809-447b-93e1-644ff417ec2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:45.695679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:45.739255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 443/875] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:45.758919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd593ae3-7b42-476b-b8a7-2b296c3e47e8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:45.865945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd593ae3-7b42-476b-b8a7-2b296c3e47e8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:45.894836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7c8f-5777-4412-acc1-6767b27e9556 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:45.895137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7c8f-5777-4412-acc1-6767b27e9556 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:45.895311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7c8f-5777-4412-acc1-6767b27e9556 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:45.898230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 444/876] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:45.906822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:45.907157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:45.907621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:45.907952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:45.908235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:45.908509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:45.908797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:45.909245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:45.910569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:45.924846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:45.925976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:45.926907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:45.930459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 433/877] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:45.931817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:45.932200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:45.933497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:45.949891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.028140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 445/878] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:46.048913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e073bde-02b0-4709-b054-5dd0bc914026 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.056535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.092009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.093139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.093634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.094224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.095253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:46.095792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.097492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.104433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 434/879] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:46.110479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.189544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.207068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676afaf-5795-41d1-bff6-9f05a2e30c14 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.243555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.282278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0228f-bfc9-44a9-8b75-931e41dd13a9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.331884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.333127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.334335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.334925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.335799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.337527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.343463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.353486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.354320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.354822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.355764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:46.356310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.357986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:46.364035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:46.365579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:46.366449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:46.375212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 435/880] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:46.376483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.376963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 446/881] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:46.378486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.561018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cceda-515d-45de-ba7d-25d6451dd8b2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.602502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf25112-ba72-4434-bcd1-5a6f88a7ae97 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.622310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.623451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.623963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.624462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.625496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] RBAC: Adding query filter params (interface=public, service_id=544aba8e28684d9ebf3691b604805c2e, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:46.626152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.627769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.635832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 436/882] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/endpoints?service_id=544aba8e28684d9ebf3691b604805c2e&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:46.643493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.643640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.643640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.643640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.643640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.644269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:46.644269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.645565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.646869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.653251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 447/883] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:46.659486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.691503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.757220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c051e-4d68-46e0-ab2a-c65c0dab6a46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.771583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4b21f-8939-4cfe-8592-2c065d1f3902 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:46.793216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.793881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.794173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.794449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.795161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.796554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67354e-39b2-41fd-8bf5-167dac7c8eab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.801826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 437/884] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/services => generated 1011 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:48:46.803726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57487fd-9ab8-4fe4-aab7-3bf6c8bc5289 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.804006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57487fd-9ab8-4fe4-aab7-3bf6c8bc5289 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.804283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57487fd-9ab8-4fe4-aab7-3bf6c8bc5289 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.805716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 438/885] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:46.805829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:46.806392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.806680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.806783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291735-6ea9-461b-8466-4c1d60971c9a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.806952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.807045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291735-6ea9-461b-8466-4c1d60971c9a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.807248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291735-6ea9-461b-8466-4c1d60971c9a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.807665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:46.807973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:46.812824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 439/886] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:46.812969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee5bfd-5cfb-440c-a353-500f0419d76c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:46.814233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.814523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.814802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.815173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 448/887] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:46.816523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:46.816860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:46.817424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:46.839272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:46.840535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:46.843680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:46.845420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:46.924909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 449/888] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:46.926007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:46.936001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 440/889] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:46.951253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.961764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:46.980465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:47.085292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d18b2-8914-4385-b13f-f9ab15cdab41 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.091219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00642d-0a07-45ba-8cfc-cad5ce815599 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.125018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.125991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.126482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.126902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.127174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.127836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.128186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.128481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:47.128599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.130700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:47.141934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:47.148319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 450/890] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:48:46 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:47.154815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:47.155853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:47.156715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 441/891] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:48:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:47.158928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:47.237202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:47.290297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ae7c-14c6-4a8c-8599-8f7313fe05e5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.336124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca780345-855d-4480-87d4-cff490790758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.337216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca780345-855d-4480-87d4-cff490790758 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.337729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca780345-855d-4480-87d4-cff490790758 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.338497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca780345-855d-4480-87d4-cff490790758 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.339481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca780345-855d-4480-87d4-cff490790758 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:47.340022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca780345-855d-4480-87d4-cff490790758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:47.341689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca780345-855d-4480-87d4-cff490790758 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:47.352801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 442/892] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:48:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:47.360094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:47.451627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:47.472724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877325-5a02-4797-bbd4-cec6cbaa7929 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.509458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.510250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.510530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.510802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.521814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:47.523284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:47.541020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:47.541987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:47.542909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:47.542909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:47.542909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:47.542909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:47.542909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 451/893] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:47.548221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca780345-855d-4480-87d4-cff490790758 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:47.661562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca780345-855d-4480-87d4-cff490790758 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.680562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:47.701957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.702999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.703504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.704002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.704982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:47.705510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:47.707108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:47.720404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 443/894] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:47.726254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:47.830896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080976d-2eff-4563-956d-4301345f48e3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:47.865320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:47.866127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:47.866408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:47.866680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:47.868592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:47.875240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:47.876607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:47.892264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:47.893236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:47.894210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 452/895] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:47.901008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:48.009938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a0011-33cd-4630-bb8e-2f0105ec8657 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:48.049086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:48.049824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:48.050113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:48.050442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:48.051309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:48.051621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:48.053239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:48.074067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 444/896] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:48.079533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:48.190097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a72c-3c6a-4cd1-9816-65c67a3bfad0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:48.233111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:48.233756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:48.234039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:48.234312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:48.300003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:48.302395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:48.303913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:48.403838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 453/897] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:48:48.409221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:48.447330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:48.522890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831953e6-4358-4452-b6ac-9f974e5492c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:48.559882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:48.560603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:48.560915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:48.561195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:48.572756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:48.574233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:48.592837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:48.593768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:48.594636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:48.594636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:48.594636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:48.594636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:48.594636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 445/898] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:48.601602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:48.701594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:48.708084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442af5f-4aa9-4bb1-865d-4856538d1b89 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:48.747576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:48.748240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:48.748502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:48.748813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:48.749542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:48.749852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:48.751221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:48.764137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 454/899] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:48.769753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:48.861479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:48.878043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a5ba-54f2-4a1c-a395-27d37c2f85a6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:48.915498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:48.916720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:48.917217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:48.917697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:48.924930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:48.926646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:48.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:48.942236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:48.943469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:48.945984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 446/900] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:48.951764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.065335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4279d4-c325-4e60-8b57-7d17b99f19f3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.105437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.106115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.106406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.106683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.107491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:49.107803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.109304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254798-c1dd-4e91-b43b-5b0abe158874 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.123978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 455/901] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:49.129822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.186346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed559-6945-4c2e-842d-4fb3a5095fdf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.186689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed559-6945-4c2e-842d-4fb3a5095fdf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.186907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed559-6945-4c2e-842d-4fb3a5095fdf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.188536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 456/902] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:49.194029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e79357-a65e-4d8d-a872-7b0ecc93b85b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.194321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e79357-a65e-4d8d-a872-7b0ecc93b85b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.194604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e79357-a65e-4d8d-a872-7b0ecc93b85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.212354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-18e79357-a65e-4d8d-a872-7b0ecc93b85b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:49.213667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18e79357-a65e-4d8d-a872-7b0ecc93b85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:49.225633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:49.250225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8d94b-3a19-4683-aa65-633588ce8bb8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.292232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 457/903] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:49.295805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef344-7fbd-4122-9125-df2a0e8cf9fd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.296297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef344-7fbd-4122-9125-df2a0e8cf9fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.296834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef344-7fbd-4122-9125-df2a0e8cf9fd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.298554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 458/904] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:49.303304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.303750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.303935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.304366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.304600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.304943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.305201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.306130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] RBAC: Adding query filter params (role.id=ee979fd898214817b97f463307d3a487, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=d04bed939eac45d2907980fea2c644d8) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:49.306446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.308155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.320654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 447/905] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/role_assignments?user.id=d04bed939eac45d2907980fea2c644d8&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=ee979fd898214817b97f463307d3a487 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:49.320654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.324707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:49.326157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:49.333756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:49.443871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cffa5c-0587-4190-bbf4-80fcd04b2de6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.448116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 459/906] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:49.470252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.485462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.486298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.486680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.487034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.487852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.489473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f201e-9d94-42a8-8033-1c4fdcfd7f0b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.508346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 448/907] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:48:49 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:49.555807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afecf0-7ce1-4ebd-a5c5-3b2cc87e003c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.556122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afecf0-7ce1-4ebd-a5c5-3b2cc87e003c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.556376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afecf0-7ce1-4ebd-a5c5-3b2cc87e003c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.558057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 449/908] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:49.563545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.563831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.564108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.583866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:49.585372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:49.589004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:49.601907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babecf16-7f95-4b99-ade9-9479183a9e17 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.648997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.650003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.650355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.650778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.651570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.653231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.657103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:49.669442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 450/909] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:49.670241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:49.671216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:48:49.672041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.674706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 460/910] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:49.681868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.690026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.803857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5715826-3223-46e6-a2d6-c5cdd6e798d7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.826537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1caeda-bdc1-462a-bed5-53ba45c7c64d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:49.843331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.844089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.844426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.844792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.845671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:49.846044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.847603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.854708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 451/911] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:49.860211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.872129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:49.873550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:49.873830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:49.874103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:49.874762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:49.876140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:49.882435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:49.897989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:49.898980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:49.899842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:49.906948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 461/912] 10.222.0.61 () {56 vars in 1132 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:49.908670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:49.955471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:50.010703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81108a2d-c0f8-4bc9-8a96-192011700424 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.045483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67336-d12d-4c66-8806-119f344562d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.060836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.061484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.061825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.062149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.062864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:50.063165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.064602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.074573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 452/913] 10.222.0.61 () {56 vars in 1114 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:50.075440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:50.092097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.093411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.093969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.094720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.096016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:50.096524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.098107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.111346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 462/914] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:50.113402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:50.180980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:50.206823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4333b2-7d4b-411b-83a3-fd29468088ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.224244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bbdc-379f-480e-80c7-68b2955947bd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.247215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:50.256493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.257401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.257837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.258253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.259136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.260677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.264832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.265804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.266098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.266466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.267275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:50.267679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.269152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.276057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 463/915] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:50.282362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:50.284556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 453/916] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:48:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:50.448995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba426-8524-4919-9013-71579faf8c56 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.498621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:50.522594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.523907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.524568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.525219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.526684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] RBAC: Adding query filter params (interface=public, service_id=d024a254bd934908b19e5ac7636ca7a3, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:50.527412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.530220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e24fcc-7a7d-41b8-97d6-53e968593c05 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.540246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 464/917] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:48:50 2022] GET /identity/v3/endpoints?service_id=d024a254bd934908b19e5ac7636ca7a3&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:50.549666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:50.656350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65018946-f965-44e6-abe5-f804487b8b0e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:50.691297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:50.691994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:50.693205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:50.693490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:50.694180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:50.695563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52230f-f1b5-495f-973c-b548bbb0878f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:50.702205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 454/918] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:50 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:48:51.604553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cf7b-86e5-4744-8790-640b5cd9c227 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:51.605131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cf7b-86e5-4744-8790-640b5cd9c227 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:51.605401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cf7b-86e5-4744-8790-640b5cd9c227 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:51.606888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 465/919] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:51.612106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f55d-1d84-41ea-8e32-bb1b09ec20f8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:51.612491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f55d-1d84-41ea-8e32-bb1b09ec20f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:51.612697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f55d-1d84-41ea-8e32-bb1b09ec20f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:51.629609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a51f55d-1d84-41ea-8e32-bb1b09ec20f8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:51.631313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a51f55d-1d84-41ea-8e32-bb1b09ec20f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:51.677543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:51.704401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 455/920] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:51.728165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91cf7b-86e5-4744-8790-640b5cd9c227 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:51.832206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91cf7b-86e5-4744-8790-640b5cd9c227 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:51.871040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:51.872178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:51.872695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:51.873206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:51.881448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4aed6-0431-4f73-ad7a-48e0b2c383b6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:51.881744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4aed6-0431-4f73-ad7a-48e0b2c383b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:51.882032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4aed6-0431-4f73-ad7a-48e0b2c383b6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:51.883957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 456/921] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:51.885469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:51.889000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:51.889294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:51.889579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:51.894797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:51.896415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:51.897305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:51.900594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 466/922] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:48:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:51.903436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:51.916651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:51.916651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:51.956214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:51.986829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 457/923] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:52.010215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.025884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa831f1-99bc-4822-bace-3cb21cf2c4c4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.063276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.063914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.064197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.064469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.065830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.065830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.066887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.075895 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 467/924] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:48:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:52.081083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.151939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12818d32-a131-42e4-9ae6-97ff2b175125 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.164065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.190398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.191482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.191482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.191752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.192401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.193939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:52.206319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:52.207345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:52.212833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 458/925] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:52.215227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.220918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931dd0e-760b-4102-965e-0e0116b3cfcf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.234411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.268780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.269941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.271338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.271808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.286013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.288164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:52.309986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:52.310995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:52.311873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:52.311873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:52.311873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:52.311873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:52.311873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 468/926] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:52.318391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.351782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb722f8-ade5-4f9f-bb84-af32ff7ba599 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.394126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.394816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.395093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.395366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.397473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.397797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.399477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.408768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 459/927] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:52.411046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.453104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5510551-35bb-49bf-a3af-baef0d4a9d75 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.454411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.497397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.498078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.498372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.498655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.499914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.500232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.501953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.520821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 469/928] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:52.523473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.546717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1df4-98de-4cd2-a181-8995689dac34 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.552516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.593770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.594480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.594771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.595049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.595790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.596100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.597589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.610859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 460/929] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:52.612115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.668082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb6e5f-c39e-49f3-99cf-7c29a15239d8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.710101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.710922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.711310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.711620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.719635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.721237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.736031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:52.740669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:52.741820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:52.742986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:52.743646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7eca-5696-4eb2-ae2a-ff230ec9eeb3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.746056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 470/930] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:52.751091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.790941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.791640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.792209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.792495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.793344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] RBAC: Adding query filter params (interface=public, service_id=dd1f24dede99401a917e930003ed3841, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.793658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.795058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0234317-7502-4f4a-95cb-39690c99d394 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.805774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 461/931] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/endpoints?service_id=dd1f24dede99401a917e930003ed3841&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:52.805774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356942b-8d30-43ff-9c0e-096e0dec0536 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.805774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356942b-8d30-43ff-9c0e-096e0dec0536 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.805774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356942b-8d30-43ff-9c0e-096e0dec0536 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.806192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 462/932] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:52.809655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356942b-8d30-43ff-9c0e-096e0dec0536 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:52.857617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:52.879424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7364c-86d8-4dc4-a638-74d8bfc83dfe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.919066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.919830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.920120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.920406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.921308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:52.921650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.923272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd8f92-db9b-4cd3-be7d-2e0a7d4d54f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.926503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356942b-8d30-43ff-9c0e-096e0dec0536 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:52.942466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 471/933] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:52.944695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffdd-7ba7-4405-b446-a21f9840ef37 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.944991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffdd-7ba7-4405-b446-a21f9840ef37 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.945275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffdd-7ba7-4405-b446-a21f9840ef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.963354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e68dffdd-7ba7-4405-b446-a21f9840ef37 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:52.964549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e68dffdd-7ba7-4405-b446-a21f9840ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:52.968295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:52.969165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:52.969599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:52.970027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:52.970954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:52.972498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:52.979390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 463/934] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:48:52.980195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.022849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.072821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 472/935] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:53.074616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf93f5-bb72-432b-8f34-0fdeefa60121 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.074911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf93f5-bb72-432b-8f34-0fdeefa60121 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.075191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf93f5-bb72-432b-8f34-0fdeefa60121 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.076771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 473/936] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:53.082540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.082833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.083263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.112987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:53.115522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:53.127201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a0c6-e288-4831-9c89-6b8d4b7ba136 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.170496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.171226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.171516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.171813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.184709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.186260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.189973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.198801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 474/937] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:53.199617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:53.205812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:53.206703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:53.207589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:53.207589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:53.207589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:53.207589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:53.207589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 464/938] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:53.213619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.304896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.318969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed3f72-f026-42b6-925d-19983d64a035 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.324985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af5ae2-906f-4ee3-9918-848df90b81f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.358124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.358782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.359063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.359576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.360034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.361478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.362881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING keystone.server.flask.common [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 21:48:53.364999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.365661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.365950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.366232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.366962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:53.367271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.368659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.381324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 465/939] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:53.381996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.sql.core [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:48:53.381996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 21:48:53.381996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [parameters: {'id': '5a4dc1904997427084d760f00157fcf0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 21:48:53.381996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80207) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:48:53.382466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a4dc1904997427084d760f00157fcf0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a4dc1904997427084d760f00157fcf0. May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:53.402339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 21:48:53.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application yield resource May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise self.value May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application session.commit() May 19 21:48:53.404176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.flush() May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application compat.raise_( May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:48:53.405128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:48:53.406044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:53.406860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [parameters: {'id': '5a4dc1904997427084d760f00157fcf0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.407705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:53.408468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a4dc1904997427084d760f00157fcf0. May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 475/940] 10.222.0.61 () {60 vars in 1140 bytes} [Thu May 19 21:48:53 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:53.412663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.472974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.495778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c75-a032-423c-9471-10b69af7d155 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.528719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b08535-f138-4908-bd9f-88c3a7e2265c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.535900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.537005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.537294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.537566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.538219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.539640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:53.552595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:53.553756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:53.559897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 466/941] 10.222.0.61 () {56 vars in 1132 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:53.561280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.561280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.577358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.578414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.578847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.579290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.587523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.589361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:53.606005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:53.606942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:53.607817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 476/942] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:53.608303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.671189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b189241-7b7a-4df0-bfde-4489b90b91d7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.709600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.710387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.710669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.710940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.718048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.719485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.726405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7eabc-4530-41d9-9b8b-2ffa8b0ce936 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:53.734752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:53.735652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:53.736546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 467/943] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:53.737525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.746820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.770331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.771392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.771858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.772306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.773223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:53.773813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.775342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.783061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 477/944] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:53.783857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.807957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:53.847111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5e80-8166-4ee1-be05-ad4083978098 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.884254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.885028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.885309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.885579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.886501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:53.886806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.888238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.898802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beac0c4-dda8-4874-85dc-8eacce73ebd1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:53.904671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 468/945] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:53.904671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:53.939881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:53.940549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:53.940901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:53.941187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:53.941946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:53.942258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:53.944191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:53.972652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 478/946] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:53.972652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:54.023443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:54.039693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557dad-7b00-42b1-80d7-b439490282f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:54.071087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:54.086390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:54.087022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:54.087411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:54.087840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:54.088741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:54.089133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:54.090653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:54.102699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 469/947] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:54.104056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:54.123635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507758a-c957-4c78-9211-d30e70e8c344 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:54.165696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:54.166950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:54.167457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:54.167933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/d04bed939eac45d2907980fea2c644d8/roles/ee979fd898214817b97f463307d3a487` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:54.170469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=d04bed939eac45d2907980fea2c644d8, role_id=ee979fd898214817b97f463307d3a487)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:54.172057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfe3f7-5db2-491c-8e7b-dbbaedad34ba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:54.192835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 479/948] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:48:53 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/d04bed939eac45d2907980fea2c644d8/roles/ee979fd898214817b97f463307d3a487 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:48:54.230499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7d53e-c550-43bd-bed5-2deb28bd2ea4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:54.282013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:54.282617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:54.282912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:54.283193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:54.283957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:54.285450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28195da5-0f9d-42be-ab8c-008fb468c7cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:54.309167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 470/949] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:48:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:55.304359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaa08-1a87-4856-bbbe-10722cd0bcd1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:55.304926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaa08-1a87-4856-bbbe-10722cd0bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:55.304926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfaa08-1a87-4856-bbbe-10722cd0bcd1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:55.306562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 480/950] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:55.312224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:55.312665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:55.312882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:55.329585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:55.330763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:55.375126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:55.412523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 471/951] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:55.435188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfaa08-1a87-4856-bbbe-10722cd0bcd1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:55.542327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:55.548037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfaa08-1a87-4856-bbbe-10722cd0bcd1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:55.583936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:55.584686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:55.584973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:55.585249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:55.585911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:55.587274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:55.600581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:55.602798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:55.605188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 481/952] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:55.608422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:55.715096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06d0b5-2b67-4b5f-aa0f-c5c10cc65a13 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:55.751851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:55.752876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:55.753384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:55.753858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:55.754788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:55.755404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:55.757012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:55.762405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 472/953] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:55.767756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:55.873138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665abdc-7e47-4abb-92eb-502e61a76360 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:55.907847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:55.908509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:55.908827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:55.909105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:55.909807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:55.910112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:55.911671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059f675-9a8d-4b63-8c79-785b64237102 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:55.917042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 482/954] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:55.922629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:55.961071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:56.042625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d0ba6-5c77-4d56-80bc-741efba7b5e8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:56.078924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:56.079928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.080436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.080943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.081923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:56.083532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c800f-981f-4501-9e03-2b149a718a3d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:56.104420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 473/955] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:48:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:48:56.403805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea30-9f3d-4b08-b4e0-362d9b651286 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.404323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea30-9f3d-4b08-b4e0-362d9b651286 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.404430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea30-9f3d-4b08-b4e0-362d9b651286 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.406038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 483/956] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:56.411892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.412334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.412521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.430327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:56.431572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:56.494658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce87b7-1136-4bcf-b88c-f54555c1b108 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.494981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce87b7-1136-4bcf-b88c-f54555c1b108 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.495266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce87b7-1136-4bcf-b88c-f54555c1b108 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.498002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 484/957] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:56.501891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.502187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.502478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.512317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 474/958] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:56.522109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:56.523734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:56.539379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:56.557435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:56.558744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:56.612666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 485/959] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:56.636027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:56.661061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a2801-5fde-4eac-afda-8a6f043ccfeb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:56.706252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:56.707173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.707604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.708684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.709659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:56.711386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:56.712946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING keystone.server.flask.common [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 21:48:56.726777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:48:56.726777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 21:48:56.726777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [parameters: {'user_id': '0804b568ebcf4d5eae170e6d9d4fbf1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 21:48:56.726777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-baf4f9ff-d504-4873-a4b5-40a9180b37a9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() May 19 21:48:56.734451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:56.735342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:48:56.736331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:48:56.737277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:48:56.738156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:48:56.739023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'user_id': '0804b568ebcf4d5eae170e6d9d4fbf1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:56.740604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 21:48:56.748333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 475/960] 10.222.0.61 () {60 vars in 1131 bytes} [Thu May 19 21:48:56 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12f437-1894-4060-ac9b-a498d68a064a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12f437-1894-4060-ac9b-a498d68a064a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.751594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12f437-1894-4060-ac9b-a498d68a064a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.753194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 476/961] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:56.753194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12f437-1894-4060-ac9b-a498d68a064a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:56.759487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064c483-6174-4f08-95ec-154ded1b0042 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:56.796607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:56.797435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.797748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.798074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.809548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8b5c67ce-8c1d-49b4-8a16-0e2fa39b14ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:48:56.818353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:56.819279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:56.820106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 486/962] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:48:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:56.820869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.821446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.821446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.838041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:56.838365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:56.839719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:56.851718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:56.867515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12f437-1894-4060-ac9b-a498d68a064a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:56.902770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:56.903569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:56.903856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:56.904139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:56.912166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:56.913719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:56.914116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 487/963] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:48:56.914960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:56.940579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:56.941612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:48:56.943120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:56.944363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 477/964] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:48:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:56.945250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.036518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569e00e-65f0-4198-8750-32285a98a620 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.053806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ef976-a9db-453e-a50e-8dc473246d50 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.072563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.073266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.073556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.073846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.074590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.074954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.076328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.084877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 488/965] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:48:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:57.085635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.096795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.097288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.098966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.099564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.099858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.100708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.102126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:57.115621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:57.117192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:57.118404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 478/966] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:57.119213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.120298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.197689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4282ff-42f1-49ad-afc3-4d8b64b82548 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.223554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bd9e-c200-4855-a7c8-7f5d434c486c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.232787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.233457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.233745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.234028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.234842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.235152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.236582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.259358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.260146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.260427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.260740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.272216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.280890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 489/967] 10.222.0.61 () {56 vars in 1101 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:57.281154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.281254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:57.302493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:57.303652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:57.304741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:57.304741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:57.304741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:57.304741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:57.305112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 479/968] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:57.311236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.360139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.385059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.415577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da16072-bf0a-4649-949d-0ef377fc0b65 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.431002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82416f11-d51b-4652-8197-6140a2982de7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.468443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.468664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.469072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.469358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.469639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.470318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.470427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.470677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.471116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.471609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.472095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.472792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] RBAC: Adding query filter params (name=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.473328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.476717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.483851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 490/969] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:57.490913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 480/970] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:57.491073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.492649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.623593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e56d-7c03-4bdd-8569-83285149f32c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.631282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c3fc-9ac8-4a4e-bc7a-157592455bca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.638256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.674583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:57.680748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.681701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.682080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.682180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.682547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.682848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.683476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.683756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.684469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.684811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.686183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.691443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 491/971] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:57.695264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.697124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:48:57.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:57.715690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:48:57.717074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:57.719728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 481/972] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:57.726076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:57.837761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec79-c41a-41aa-9df6-dfc7b8b4b712 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:57.881350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:57.881884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:57.882794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:57.882794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:57.883890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:57.883890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:57.885866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:57.904121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 492/973] 10.222.0.61 () {56 vars in 1149 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:57.909811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.012453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df28b6a-c5ba-43a4-8683-0fe7b18333fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.055817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.056776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.057205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.057652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.104523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.131706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.133209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.236797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 482/974] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:48:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:48:58.239669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.300286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.350001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af981b7d-2a1e-4f5f-8c7b-472f51fa1258 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.386903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.387698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.387984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.388264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.398999 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.400441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6fc9e45e-ea87-4452-8a95-fad96fb7d647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:48:58.417977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:58.419335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:58.420208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:48:58.420208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:48:58.420208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:48:58.420208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:58.420208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 493/975] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:48:58.426394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.443634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a523-f9b8-454e-827c-2d90d4e37d21 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.444067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a523-f9b8-454e-827c-2d90d4e37d21 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.444350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a523-f9b8-454e-827c-2d90d4e37d21 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.445936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 494/976] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:48:58.450938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.451231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.451512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.468687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:58.469873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:48:58.508329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.540712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e6f9-1e6a-47af-a92c-f4e88fea2a62 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.547085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 495/977] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:48:58.570400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.577635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.578671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.579177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.579653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.580719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:58.581244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.583438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.596120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 483/978] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:58.601695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.607902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.681150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ee6c9-72e5-40b6-90cf-e4aec4b0be7e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.701882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f28f9e-04b1-4a22-9252-b85736dd0994 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.717825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.718624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.718909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.719188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.719840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.721221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:58.734034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:58.735047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:48:58.735874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 496/979] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:58.741580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.742489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.743296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.743582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.743859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.745890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.751918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.753780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:48:58.768710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:48:58.769590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:48:58.771418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 484/980] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:48:58.776087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.858696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:58.863229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a7ca-b2d9-4f27-aed3-f8e1e90fb979 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.884892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eed84-d2c5-4eed-bd09-cc8e17a4ac69 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:58.902193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.902972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.903258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.903650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.904364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:58.904721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.906079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.917033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 497/981] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:58.917919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:58.922331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:58.923004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:58.923294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:58.923608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:58.924442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:58.924790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:58.926251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:58.940478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 485/982] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:58.946503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:59.009522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:59.032277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29b66-9a0e-4b9e-98bb-a292a7936247 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:59.069155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:59.069806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.070095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.070374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.071085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:59.071395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:59.072284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b2d6-84b1-46af-ae9d-5406808a98a4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:59.072807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:59.083380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 498/983] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:48:59.084481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:59.112875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:59.113464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.113752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.114076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.115141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:59.115510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] RBAC: Adding query filter params (role.id=ee979fd898214817b97f463307d3a487, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=d04bed939eac45d2907980fea2c644d8) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:59.115825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:59.117353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:59.123488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 486/984] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:48:58 2022] GET /identity/v3/role_assignments?user.id=d04bed939eac45d2907980fea2c644d8&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=ee979fd898214817b97f463307d3a487 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:59.197227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36542-eab0-47d8-b975-84a0461071c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:59.234582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:59.235727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.236209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.236706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.237716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] RBAC: Adding query filter params (interface=admin, service_id=dd1f24dede99401a917e930003ed3841, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:48:59.238253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:59.240712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fe33-b69f-4e19-a04a-f32e14452d50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:59.251392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 499/985] 10.222.0.61 () {56 vars in 1249 bytes} [Thu May 19 21:48:59 2022] GET /identity/v3/endpoints?service_id=dd1f24dede99401a917e930003ed3841&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:48:59.252771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:48:59.362489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b670454-0df6-4955-b738-c6cd8a5c9245 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:48:59.382731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:48:59.401324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:48:59.402001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.402286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.402562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.403248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:48:59.404710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca39f12-19f9-4b10-bbc0-6bbb10d710cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:48:59.411668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 487/986] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:48:59 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:48:59.951910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718f83-40ad-46bc-80a8-1b0dc2451ab8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.952425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718f83-40ad-46bc-80a8-1b0dc2451ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.952504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718f83-40ad-46bc-80a8-1b0dc2451ab8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.954043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 500/987] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:48:59.959039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4642d-72f8-47ec-9ace-e00780355c65 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:48:59.959424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4642d-72f8-47ec-9ace-e00780355c65 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:48:59.959608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4642d-72f8-47ec-9ace-e00780355c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:48:59.977082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-36f4642d-72f8-47ec-9ace-e00780355c65 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:48:59.978848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36f4642d-72f8-47ec-9ace-e00780355c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:00.053135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 488/988] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:48:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:00.074728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39718f83-40ad-46bc-80a8-1b0dc2451ab8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:00.166087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:00.176591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39718f83-40ad-46bc-80a8-1b0dc2451ab8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:00.209493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:00.210072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:00.210362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:00.210637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:00.211362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:00.212727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80714e70-7eb5-4053-9aa7-32dcff1c1b09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:00.228256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 501/989] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:49:00 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:01.430535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994328-f951-4284-ac0d-8d2e8c98a4ce None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.431039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994328-f951-4284-ac0d-8d2e8c98a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.431146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994328-f951-4284-ac0d-8d2e8c98a4ce None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.432597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 489/990] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:01.437796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f28736-f184-4644-b32d-2bbf0f118e43 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.438178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f28736-f184-4644-b32d-2bbf0f118e43 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.438357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f28736-f184-4644-b32d-2bbf0f118e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.456021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a5f28736-f184-4644-b32d-2bbf0f118e43 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:01.457234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5f28736-f184-4644-b32d-2bbf0f118e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:01.539003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 502/991] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:01.561679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1994328-f951-4284-ac0d-8d2e8c98a4ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:01.661280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71d73-8e2d-42f8-bafe-f231b0135cae None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.661627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71d73-8e2d-42f8-bafe-f231b0135cae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.661846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71d73-8e2d-42f8-bafe-f231b0135cae None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.663363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 503/992] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:01.668022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.668322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.668610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.669120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1994328-f951-4284-ac0d-8d2e8c98a4ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:01.684110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:01.686557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:01.687748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:01.707471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:01.708801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.709087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.709365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] PATH_INFO: `/v3/services/image` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.710021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:01.711408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:01.723367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:01.725059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:01.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 490/993] 10.222.0.61 () {56 vars in 1105 bytes} [Thu May 19 21:49:01 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:01.733557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:01.768668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 504/994] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:01.790156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:01.800985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:01.840734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0241a7-b903-4ce9-96d7-d1a704229b91 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:01.877237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:01.877904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.878192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.878472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.879190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:01.879532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:01.881088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:01.886283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 491/995] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:01.891551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:01.907161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60751bff-00fc-43dd-b075-0fd986d04597 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:01.935089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:01.945881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:01.947059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:01.947531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:01.947994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:01.948865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:01.950618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:01.964147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:01.965288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:01.971363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 505/996] 10.222.0.61 () {56 vars in 1126 bytes} [Thu May 19 21:49:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:01.972192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:02.001823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67663d-c30d-44d1-8e77-a92b964665f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.043908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.044701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.045081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.045342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.046066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] RBAC: Adding query filter params (type=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:02.046377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.047773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.052641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:02.061642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 492/997] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:02.097645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec30a4-2a3c-4e7b-9862-0b62102981ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.136902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.137592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.137889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.138171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.138899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:02.139211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.140599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.151102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 506/998] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:49:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:02.152493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:02.263821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39179-d60d-4a94-80a7-dc1e226de278 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.266137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:02.310655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.311349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.311638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.311915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.312665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:02.312976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.314417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc2509-3054-4d9b-85bd-d6468957e90b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.320638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 493/999] 10.222.0.61 () {56 vars in 1122 bytes} [Thu May 19 21:49:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:02.326531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:02.437028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34030a-3f42-40f2-a2e7-e0c1b3972f8e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.471560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.472220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.472493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.472803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.474764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.476197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.480104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:02.485347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941df65-d0b4-4fc4-b843-6fdc65c171fa None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.485833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941df65-d0b4-4fc4-b843-6fdc65c171fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.486157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941df65-d0b4-4fc4-b843-6fdc65c171fa None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.487833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 494/1000] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:49:02.493753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.494050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.494335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.501103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 507/1001] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:49:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:02.514945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:02.516092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:02.601785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 495/1002] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:02.624145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:02.737998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a33-6137-4c3c-9741-f74325ded6a9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.774653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.775365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.775654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.775935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.777164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.779961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:02.791954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:02.795363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:02.801050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 508/1003] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:49:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:02.805106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:02.900967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:02.907714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccacf52-4118-4c8f-b90f-8ad4590da3a1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:02.944196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:02.944900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:02.945185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:02.945462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:02.946174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:02.946481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:02.947873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:02.957460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 496/1004] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:49:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:02.959128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:03.034561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:03.079112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8fba8-355b-44e4-ad9d-2086f874ccec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:03.114093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:03.114762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:03.115051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:03.115332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:03.116040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:03.116353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:03.117815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:03.122842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 509/1005] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:49:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:03.128465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:03.231099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d19f6-ce3a-4ac9-bc0d-2830c0a3a5d2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:03.266223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:03.266883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:03.267178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:03.267459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:03.268254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] RBAC: Adding query filter params (interface=public, service_id=aa716c9da3704997b52445ef03d65301, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:03.268568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:03.269988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d481016-7de2-4a65-a871-8d4874fb5503 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:03.275530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 497/1006] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:49:03 2022] GET /identity/v3/endpoints?service_id=aa716c9da3704997b52445ef03d65301&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:03.281268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:03.384329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e56dc-73ab-433f-afe9-b97dd8c21e8d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:03.406517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:03.419408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:03.420069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:03.420357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:03.420656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:03.421343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:03.422708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781aa62b-230e-4dc2-a82f-29df74a02584 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:03.432841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 510/1007] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:49:03 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:49:04.328104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e41-fbf1-49d0-8a4f-b1fa3e24f896 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:04.328869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e41-fbf1-49d0-8a4f-b1fa3e24f896 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:04.328869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e41-fbf1-49d0-8a4f-b1fa3e24f896 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:04.330255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 498/1008] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:49:04.335478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccabc8-8609-473b-8de3-a3b357e33f7e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:04.335867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccabc8-8609-473b-8de3-a3b357e33f7e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:04.336040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccabc8-8609-473b-8de3-a3b357e33f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:04.354279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-09ccabc8-8609-473b-8de3-a3b357e33f7e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:04.355466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-09ccabc8-8609-473b-8de3-a3b357e33f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:04.425132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 511/1009] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:04.452495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e99e41-fbf1-49d0-8a4f-b1fa3e24f896 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:04.477713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:04.558611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e99e41-fbf1-49d0-8a4f-b1fa3e24f896 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:04.593450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:04.594033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:04.594323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:04.594599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:04.595327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:04.596740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88303c7e-0953-43e2-99cb-1d783ac7af4c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:04.611132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 499/1010] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:49:04 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:04.826008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbde84-3a2c-4efb-9cc2-9d7ace0a8e3a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:04.826327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbde84-3a2c-4efb-9cc2-9d7ace0a8e3a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:04.826577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbde84-3a2c-4efb-9cc2-9d7ace0a8e3a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:04.828258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 512/1011] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:04.833345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:04.833710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:04.833920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:04.851790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:04.853094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:04.930564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 500/1012] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:04 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:04.954200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbde84-3a2c-4efb-9cc2-9d7ace0a8e3a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:05.062122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbde84-3a2c-4efb-9cc2-9d7ace0a8e3a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:05.080461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:05.100836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:05.101631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:05.101971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:05.102198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:05.103378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:05.105107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7568f7-6ab7-4e42-8e22-0e367e82dfcf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:05.111391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 513/1013] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:05.112488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:05.124106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:05.218930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe521-ea53-4cdc-8085-af1514f027ca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:05.253397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:05.254067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:05.254906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:05.255220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:05.256185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:05.257647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4cfe-03c5-4550-87f4-fcc06a1c2904 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:05.283563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 501/1014] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:49:05 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:05.688719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1687-6ad2-437d-8452-1474c796ff13 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:05.689252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1687-6ad2-437d-8452-1474c796ff13 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:05.689329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1687-6ad2-437d-8452-1474c796ff13 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:05.690789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 514/1015] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:05.695960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:05.696378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:05.696555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:05.714463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:05.715665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:05.776004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:05.792778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 502/1016] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:05 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:05.815760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1687-6ad2-437d-8452-1474c796ff13 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:05.923520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1687-6ad2-437d-8452-1474c796ff13 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:05.958409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:05.959172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:05.959487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:05.959772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:05.960427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:05.961813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:05.974275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:05.975264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:05.976132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 515/1017] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:49:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:05.983253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:06.094258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf400e7-6658-475a-9491-0aa57d4e42cd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:06.128799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:06.130003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:06.130508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:06.130837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:06.131551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:06.131859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:06.133284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:06.143035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 503/1018] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:49:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:06.143975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:06.195660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:06.247978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515c18-9dcd-43cf-a543-69203cc9990e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:06.281444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:06.282107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:06.282397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:06.282682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:06.283385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:06.283697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:06.285105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aba562-1ffb-4288-8b30-9ae2462e519b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:06.294992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 516/1019] 10.222.0.61 () {56 vars in 1112 bytes} [Thu May 19 21:49:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:06.296105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:06.352394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:06.399376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33275481-436e-4691-9c10-ac1f3a9b1f17 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:06.435847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:06.436512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:06.436827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:06.437113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:06.437868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:06.439255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b82677-b25e-4117-9a8e-33d9e4cf4418 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:06.462545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 504/1020] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:49:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:06.924663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cde3d-0356-4682-b7b2-e69edab453e4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:06.925209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cde3d-0356-4682-b7b2-e69edab453e4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:06.925209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cde3d-0356-4682-b7b2-e69edab453e4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:06.927284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 517/1021] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:06.932555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:06.933000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:06.933182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:06.950032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:06.951217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:07.036698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 505/1022] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:07.053625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cde3d-0356-4682-b7b2-e69edab453e4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.158703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cde3d-0356-4682-b7b2-e69edab453e4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.172100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:07.196009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.196839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.197131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.197415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.198074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.199469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:07.211825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:07.212832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:07.218721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 518/1023] 10.222.0.61 () {56 vars in 1105 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:07.220987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.258664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:07.327040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8adb7-664d-4d9e-b1c9-c95cfd15a2fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.362019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.362685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.362964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.363235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.364600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:07.364945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.366326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.376187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 506/1024] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:07.377485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.486855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed75a-7593-49ed-bb9a-f5b77e463732 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.520810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.521475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.521756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.522033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.522739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:07.523047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.524403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99670-c84f-4d96-b605-56e1e721f026 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.534269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 519/1025] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:07.535170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.574216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126061f-be2b-493b-939a-75e8fb18f4c9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.574533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126061f-be2b-493b-939a-75e8fb18f4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.574776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126061f-be2b-493b-939a-75e8fb18f4c9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.576146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 520/1026] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:07.581092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.581386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.581666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.599060 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:07.600275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:07.638752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:07.639750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:07.643382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bb24f-e7f6-490c-9d1c-acb651227a9a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.673365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 521/1027] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:07.683711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.684442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.684814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.685108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.685928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] RBAC: Adding query filter params (interface=public, service_id=fa99849f09814d338c68df42b5f9ce9b, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:07.686242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.687690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.698750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 507/1028] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/endpoints?service_id=fa99849f09814d338c68df42b5f9ce9b&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:07.699725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.700079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.809502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174e9b7-ae10-421e-b279-6f96b77b5666 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.812496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef179abf-83b9-41e8-b38a-8ab51215de79 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:07.848330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.849501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.849963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.850390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:07.850609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.851185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:07.851471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:07.851752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:07.852108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.852987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:07.854166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ced871-6962-4040-b3e2-c56bf7766c41 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.854663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:07.860660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 522/1029] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:07.861942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:07.876646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 508/1030] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:49:07 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:49:07.913127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:07.982860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4052-2a9a-4810-a09b-916b5d626630 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:08.029811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:08.030549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:08.030790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:08.031070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:08.031803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:08.033469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b36c59-34ae-448d-81ad-e383e2d9a2f6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:08.061233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 523/1031] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:49:07 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:49:08.829191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619fd29-b7c6-4ec4-a5a0-8df83f506ff0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:08.829748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619fd29-b7c6-4ec4-a5a0-8df83f506ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:08.829748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619fd29-b7c6-4ec4-a5a0-8df83f506ff0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:08.831244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 509/1032] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:08.836547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:08.836878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:08.837160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:08.855121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:08.856314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:08.924841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 524/1033] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:08.947991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619fd29-b7c6-4ec4-a5a0-8df83f506ff0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:08.976935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:09.062085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619fd29-b7c6-4ec4-a5a0-8df83f506ff0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.097678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.098925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.099410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.099892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.110541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:09.118867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:09.119773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:09.120604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:09.121263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 510/1034] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:49:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:09.127766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:09.154968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:09.234324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb500a-feab-4c62-80dd-56bda872dd26 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.268214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.268943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.269218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.269494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.270930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:09.271240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:09.272661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:09.286154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 525/1035] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:09.287392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:09.378029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:09.391200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122968ae-e2ce-456a-809e-f3408ba8d1c2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.425054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.425853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.426704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.427046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.437753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:09.439219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:09.456413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:09.457868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:09.458768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:09.458768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:09.458768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:09.458768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:09.458768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 511/1036] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:09.465904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:09.555563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:09.571929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50baa9e-8da0-49af-a1b3-b6d975bbb335 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.607482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.608149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.608438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.608752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.609484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:09.609796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:09.611160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:09.627587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 526/1037] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:09.629105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:09.733498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c06108-6c5f-4add-aa1a-d565671c17bf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.767488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.768282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.768567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.768892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.777225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:09.778696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:09.781336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:49:09.794332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:09.795299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:09.796280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 512/1038] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:09.803234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:09.905443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e008758-c1cb-4018-8587-45b5c27dc585 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:09.943355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:09.944048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:09.944330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:09.944606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:09.945500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:09.945811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:09.947272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d83dfa-0886-4a68-895a-e328e494a78d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:09.971495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 527/1039] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:09.972668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.085653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c79466-e165-44d2-a309-b2eb1dafbdb9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.119722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.120375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.120678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.120954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.125559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0393b-434e-4184-b54b-c256b95d11fb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.125857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0393b-434e-4184-b54b-c256b95d11fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.126137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0393b-434e-4184-b54b-c256b95d11fb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.127651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 528/1040] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:10.133421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.133979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.133979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.139809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.152741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:10.154290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:10.190591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.194012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.195978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.236669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 529/1041] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:10.257973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.310881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 513/1042] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:10.316561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.357458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d07a5-8942-48a1-8c83-4233c827157f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.392300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.393119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.393403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.393676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.394327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.395726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.397565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-45db95a7-aa32-4fdd-ab19-c9be50c15e79 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:10.409218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:10.410229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:49:10.411827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:10.412578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 530/1043] 10.222.0.61 () {56 vars in 1105 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:10.414834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e67b-0412-4683-a9de-6c6fb5845fbb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.415344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e67b-0412-4683-a9de-6c6fb5845fbb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.415859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e67b-0412-4683-a9de-6c6fb5845fbb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.418672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 531/1044] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:10.420054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e67b-0412-4683-a9de-6c6fb5845fbb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.421826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.425467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf844b1-e56d-4189-8857-4f0bac230151 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.460062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.460809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.461087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.461369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.471212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.472693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4af30ce4-c4f1-48ed-b77b-ada586211751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:10.491593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:10.492499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:10.493439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:10.493439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:10.493439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:10.493439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:10.493439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 514/1045] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:10.494152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.494433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.494705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.510633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:10.511802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:10.535445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e67b-0412-4683-a9de-6c6fb5845fbb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.570449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.571187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.571537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.571881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.572585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:10.572943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.574268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.576214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 515/1046] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:10.577060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.579397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 532/1047] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:10.589811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.651716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.658911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.678766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19247257-2d4b-4c26-9c3f-7741c2791958 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.693607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8afd-1d3f-416b-b782-2320dbdd18ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.712929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.713629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.713986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.714341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.715133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:10.715449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.716865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.732328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 516/1048] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:10.732946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.733176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.734055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.734423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.734706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.735447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:10.735754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.737145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.747941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 533/1049] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:10.747941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.836588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c788-0463-4999-809d-07ab8b9948ff None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.848168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead5249-8389-4d90-b6fe-3d1c1f35721f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:10.871548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.872341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.872643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.872934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.875382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.884222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:10.885052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:10.885834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:10.886114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:10.886385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:10.891870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:10.892794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:10.893631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 517/1050] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:10.894328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:10.894405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.895109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:10.898861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:49:10.911635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:10.912557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:10.913639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:10.916937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 534/1051] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:10.917590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:10.996979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c453-1e24-42b6-bfdb-e84e82d3cc09 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.016889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb44d4-3274-471f-ad99-435408958d95 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.031916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.032576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.032892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.033252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.033907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.035292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.053361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.054026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.054321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.054610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.055341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:11.055650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.057042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.057755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 518/1052] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:49:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:11.058868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.068087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 535/1053] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:11.074146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.134061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:11.134371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:11.195227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ecc5d-4c0f-4dd9-a1cf-e03541528f2a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.195734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7eb37-26a2-4187-a0e7-fe318d29f3ef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.235075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.236006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.236357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.236569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] PATH_INFO: `/v3/users/ef05eeb193d14e04946c9f38a54087f5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.239639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.240306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.240593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.241736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.241736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:11.242031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.243507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.254329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] RBAC: Authorizing `identity:get_user(user_id=ef05eeb193d14e04946c9f38a54087f5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.256296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.257519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 519/1054] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:11.262651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 536/1055] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/users/ef05eeb193d14e04946c9f38a54087f5 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:11.263126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.263588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.373079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c9b9-6ad8-44eb-82bf-68965c8a79fa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.376038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a59dd7-ed32-4b5e-b2a8-6e7c6e8316e0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.380671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:11.394474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:11.410625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.412046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.412335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.412611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.418920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.419582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.419872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.420154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.421132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] RBAC: Adding query filter params (role.id=ee979fd898214817b97f463307d3a487, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:11.421445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.422930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.437047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 520/1056] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=ee979fd898214817b97f463307d3a487 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:11.487621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.489283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.601511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 537/1057] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:11.602872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.707085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf37fb9-c5e7-423b-adf4-64e3f64c8e4c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.742960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.743823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.744106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.744387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.745678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.747133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838acff-f9c4-42ca-ae6d-b78547ca83cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.749049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 521/1058] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:11.755663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:11.763900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:11.860035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9314-3db5-43b8-89f5-581899335fa4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:11.896104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:11.896836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:11.897136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:11.897425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:11.898310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=3a496fa70bbf4c70819d5a7e212232c1, user.id=ef05eeb193d14e04946c9f38a54087f5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:11.898648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:11.900089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8d7a-996e-400d-b936-33d38b41aef6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:11.912658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 538/1059] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:11 2022] GET /identity/v3/role_assignments?user.id=ef05eeb193d14e04946c9f38a54087f5&scope.project.id=3a496fa70bbf4c70819d5a7e212232c1&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:13.335508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602214ca-b410-4734-981b-af25c57ee9a7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.336050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602214ca-b410-4734-981b-af25c57ee9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.336050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602214ca-b410-4734-981b-af25c57ee9a7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.337498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 522/1060] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:13.342658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255d65-d16e-4da1-9e96-f71822b537d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.343045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255d65-d16e-4da1-9e96-f71822b537d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.343203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255d65-d16e-4da1-9e96-f71822b537d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.360695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-32255d65-d16e-4da1-9e96-f71822b537d5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:13.361871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32255d65-d16e-4da1-9e96-f71822b537d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:13.425177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:13.433261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 539/1061] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:13.453005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602214ca-b410-4734-981b-af25c57ee9a7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:13.506670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:13.562919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602214ca-b410-4734-981b-af25c57ee9a7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:13.599703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:13.600737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.601216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.601690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.658053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57047-88ef-4b2d-8834-c02e89d4163c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.658373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57047-88ef-4b2d-8834-c02e89d4163c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.658643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57047-88ef-4b2d-8834-c02e89d4163c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.660161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 540/1062] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:13.665310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.665599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.665881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.680135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:13.681643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:13.683215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:13.684391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:13.731663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:13.755536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 541/1063] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:13.778165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:13.789514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 523/1064] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:13.795266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:13.889808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237e7f1-9972-422d-87a5-9efc1f6879b5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:13.902335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862f5a-be74-4188-aa5c-90de81d88834 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:13.924691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:13.933040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:13.933876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.934162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.934441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.940929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:13.941881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:13.942161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:13.942441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:13.947091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:13.954815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:13.955489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:13.956433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:13.957940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:13.958755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:13.961321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 542/1065] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:49:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:13.963361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:13.967144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:13.980385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:13.981400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:13.982294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:13.982294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:13.982294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:13.982294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:13.989606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 524/1066] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:13.991132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.080571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd184a-e65a-40f3-8588-0e9e1967fa5b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.101317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf893-1bf7-496e-8824-e83b30c8e66f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.117496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.118155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.118437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.118713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.119485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:14.119954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.121344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35305c-8b39-4ae7-b1fd-8bc50bc3c774 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.130212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 543/1067] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:49:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.131493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91298-f069-4ee3-95d9-150efedacb5c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.131770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91298-f069-4ee3-95d9-150efedacb5c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.132055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91298-f069-4ee3-95d9-150efedacb5c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.133294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 544/1068] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:14.135587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91298-f069-4ee3-95d9-150efedacb5c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.143164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.143766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.144049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.144329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.145125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:14.145438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.146816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828c655-c4b0-4066-b5bb-59322a4c4a59 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.158019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 525/1069] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.159212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.159503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.159784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.178488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:14.179732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:14.204829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.239783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.257387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 526/1070] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:14.265804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91298-f069-4ee3-95d9-150efedacb5c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.279751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.306094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.306887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.307213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.307435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.319227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.320768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:14.340375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:14.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:14.342346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:14.342346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:14.342346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:14.342346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:14.342346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 545/1071] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:14.347234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.389508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5ef73-845d-486e-b5cb-4c09f231f9a5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.427872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.428602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.428942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.430101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.448441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:14.456090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:14.457110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:14.458168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:14.458807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 527/1072] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:14.464746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.470253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45598eb0-c76c-4f01-ac17-1f21e174de75 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.474428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.516705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.517372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.517657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.517934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.518540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.518665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:14.518952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.520786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.532718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 546/1073] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.538990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.571956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61319ad-3451-4ef7-b087-a9288e298c3c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.604839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.605499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.605784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.606060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.606804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:14.607109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.608498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.617272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 528/1074] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.623029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.639572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36151b9d-6e11-4cc3-94c4-c5bc01011d0b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.675606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.677220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.677536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.677844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.685691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.687066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:49:14.701406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:14.702534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:14.703701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 547/1075] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:14.708345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.733420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.740812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d1ab7-d7e2-4e9b-a329-85ab18de851f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.751025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.776019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.776909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.777196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.777469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] PATH_INFO: `/v3/users/ef05eeb193d14e04946c9f38a54087f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.778663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=ef05eeb193d14e04946c9f38a54087f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.780110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.781772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 529/1076] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/users/ef05eeb193d14e04946c9f38a54087f5 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.788392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.809110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc176b-f822-4a5a-bca7-76f6501f28e9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.841988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.842653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.842941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.843225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.844070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:14.844388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.845849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.864018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 548/1077] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:14.869774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.889542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc7e05-480e-4dfb-8e69-219f78b54eb3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.923432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:14.924286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:14.924574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:14.924885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:14.926783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:14.928164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:14.929911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 530/1078] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:14.935518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:14.951753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:14.974305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0429a1c-1078-4d44-b85b-70df57286cf6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:14.983731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:15.009828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.010628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.010911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.011190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.021539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.022977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.039280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb1844-d7c6-49b6-a096-87fb27884c11 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:15.041630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:15.042579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:15.043511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:15.043511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:15.043511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:15.043511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:15.043511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 549/1079] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:15.048427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:15.074059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.074936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.075219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.075493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1/users/ef05eeb193d14e04946c9f38a54087f5/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.077495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a496fa70bbf4c70819d5a7e212232c1, user_id=ef05eeb193d14e04946c9f38a54087f5, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.078940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.091273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 531/1080] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:49:14 2022] PUT /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1/users/ef05eeb193d14e04946c9f38a54087f5/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:49:15.156265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654d98-40be-4770-97fe-6d5340f102e1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:15.194341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.195025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.195315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.195598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.196337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:15.196675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.198090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.200018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:15.212562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 550/1081] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:15.218342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:15.326248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347ff8b-d591-49ea-bb1c-d3f1bff5e5c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:15.333203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:15.360345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.361157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.361439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.361709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.368783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.370234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:49:15.384759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:15.385853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:15.386777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 532/1082] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:15.391690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:15.494177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803833-030c-4076-b5a3-0e0b74082afc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:15.530690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.531354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.531653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.531934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.532764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:15.533082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.534514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe21eb-309c-49e0-a13e-addf18bc971f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.547419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 551/1083] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:15.553653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:15.653506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acfac8-b141-4a7e-98fb-6ff8b4860813 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:15.686484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:15.687365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:15.687653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:15.687929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/ee979fd898214817b97f463307d3a487` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:15.690170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=6b6fbc800c444f2aa44f2c33c40149f0, role_id=ee979fd898214817b97f463307d3a487)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:15.692114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931f5cd-8121-40d8-b1a4-c0ee587869b9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:15.705238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 533/1084] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:49:15 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/6b6fbc800c444f2aa44f2c33c40149f0/roles/ee979fd898214817b97f463307d3a487 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:49:17.346768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bb09-92a3-463f-96a0-79a44b9dc7e8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.347319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bb09-92a3-463f-96a0-79a44b9dc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.347319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bb09-92a3-463f-96a0-79a44b9dc7e8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.349235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 552/1085] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:17.354658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.355083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.355340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.373176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:17.374367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:17.394298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:17.443435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 534/1086] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:17.466981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07bb09-92a3-463f-96a0-79a44b9dc7e8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:17.525289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:17.572951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07bb09-92a3-463f-96a0-79a44b9dc7e8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:17.607828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:17.608543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.608860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.609139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.619642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:17.627166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:17.628374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:17.629718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 553/1087] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:17.634490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:17.732443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d82a2b-368e-4347-afb6-37311ebdfbc5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:17.769478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:17.771516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.771800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.772074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.772898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:17.773206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:17.774635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:17.783613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 535/1088] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:17.789450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:17.890637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d80-1385-4be7-ac30-d28ab563e7be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:17.905104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:17.925523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:17.926394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.926677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.926950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] PATH_INFO: `/v3/users/ef05eeb193d14e04946c9f38a54087f5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.928122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef05eeb193d14e04946c9f38a54087f5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:17.929556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86757921-27a6-4520-86cb-f03308fe1df0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:17.931202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 554/1089] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:17 2022] GET /identity/v3/users/ef05eeb193d14e04946c9f38a54087f5 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:17.937339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:17.938428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30854457-ab5e-4990-824d-2c8aca0f13c2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.938808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30854457-ab5e-4990-824d-2c8aca0f13c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.939147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30854457-ab5e-4990-824d-2c8aca0f13c2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.940501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 555/1090] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:49:17.946282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:17.946569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:17.946849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:17.958133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:17.964144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:17.965394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:18.035562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 556/1091] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:18.043363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7156-a0c5-44e4-9027-0e676c236926 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.059351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.080184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.080184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.080184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.080184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.147079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.148477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.160143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2aeee-2261-4083-8bd4-8aba6e82b7e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.166467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.174232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.196611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.197438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.197871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.198146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.207940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:18.216048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:18.217037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:18.217897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 557/1092] 10.222.0.61 () {56 vars in 1120 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:18.223175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.250831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 536/1093] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:18.257200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.325016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2134a-35e8-44be-8680-5c499cb764fa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.357075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf971e1-8da7-4ef6-966c-68186a3ee092 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.357738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.358398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.358682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.358959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.359697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:18.360002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.361399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.370312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 558/1094] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:18.375862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.375989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.394109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.395072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.395357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.395633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.396842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.398341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.400175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 537/1095] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:18.405776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.411416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.477970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5d30-b243-4089-817a-86352dc53de4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.503995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19c67-6886-4686-8f24-5ad95d0e1232 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.511451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.512255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.512674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.512961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.524019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.525510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.540882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.541528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.541810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.542085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.542913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=3a496fa70bbf4c70819d5a7e212232c1, user.id=ef05eeb193d14e04946c9f38a54087f5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:18.543344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:18.544216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:18.545275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:18.546223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:18.546223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:18.546223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:18.546223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:18.546470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.546552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 559/1096] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:49:18.551574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.552722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 538/1097] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/role_assignments?user.id=ef05eeb193d14e04946c9f38a54087f5&scope.project.id=3a496fa70bbf4c70819d5a7e212232c1&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:18.643381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.659281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d0724-492f-44f2-8162-97b50518b68d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.696100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.696819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.697109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.697390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.698139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:18.698454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.699874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.713005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 560/1098] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:18.718753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.772025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:18.821872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f19a-d9a3-467a-b697-39449058c2ca None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:18.855671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:18.856445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:18.856758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:18.857031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:18.864646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:18.866100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:49:18.880299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:18.881362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:18.882360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 539/1099] 10.222.0.61 () {56 vars in 1093 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:18.887132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:18.985715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108778-73b6-4753-9d03-0e56e594cb9d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.020833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.021511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.021802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.022085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.022935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:19.023249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.025015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.043201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 561/1100] 10.222.0.61 () {56 vars in 1133 bytes} [Thu May 19 21:49:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:19.048814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.148046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc6fc7-3d32-465c-bf35-de02e3aacdb7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.158683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.183404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.184051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.184338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.184632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.250018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.251414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.349844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 540/1101] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:49:19.355545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.359309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.455934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ddd83-0812-40d7-abe1-ba370bdc7161 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.489300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.490086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.490374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.490656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.500492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.501908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:19.520390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:19.521567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:19.522485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:49:19.522485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:19.522485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:49:19.522485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:19.522485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 562/1102] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:19.527304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.538783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.623972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990ed19-f33c-442b-9383-566e32a6d3a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.657896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.658548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.658836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.659114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.659834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:19.660179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.661932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.673158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 541/1103] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:19.679135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.741504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.785953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282ae0-b52e-4e0c-b39c-77e5002006bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.819761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.820551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.820870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.821152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.828257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.829845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:49:19.844332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:19.845446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:49:19.846557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:19.848764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 563/1104] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:19.851813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.920979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.952420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe71850-5ebb-4761-a4c6-5ff36e936213 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.986423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.987085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.987396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.987698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.988584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:19.988919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.990375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943598f8-a28a-4f56-a6bb-d1d514bcec83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:20.002510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 542/1105] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:20.009008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:20.112393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291871-c9a9-4470-9369-0952aa75d7d4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:20.128578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:20.148369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:20.149053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:20.149343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:20.149622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:20.150463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] RBAC: Adding query filter params (role.id=ee979fd898214817b97f463307d3a487, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=6b6fbc800c444f2aa44f2c33c40149f0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:20.150781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:20.152199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0f620-5a20-4701-8100-40d7bc553418 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:20.158054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 564/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:20 2022] GET /identity/v3/role_assignments?user.id=6b6fbc800c444f2aa44f2c33c40149f0&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=ee979fd898214817b97f463307d3a487 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:20.810096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823d4f-3b66-4427-89f5-fe7f17e0b350 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:20.810623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823d4f-3b66-4427-89f5-fe7f17e0b350 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:20.810742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823d4f-3b66-4427-89f5-fe7f17e0b350 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:20.812167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 543/1107] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:20.817887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:20.818294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:20.818455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:20.835927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:20.837143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:20.908162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 565/1108] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:20.932032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d823d4f-3b66-4427-89f5-fe7f17e0b350 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:21.036649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d823d4f-3b66-4427-89f5-fe7f17e0b350 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:21.064783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:21.071215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:21.072003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:21.072289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:21.072569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:21.073776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:21.075224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09b4a1-4230-4422-9ef4-5eedab4feb8c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:21.076884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 544/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:21.083498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:21.182613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a2e8f-a3cb-4be8-bbbc-78945f51dc9c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:21.193770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:21.216679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:21.217339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:21.217625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:21.217905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:21.218622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:21.220041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3318034-494c-44f9-b8e3-9d12084d8d03 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:21.240008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 566/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:49:21 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:49:23.477836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89003-039a-4322-be4d-4ae64e424c53 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:23.478647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89003-039a-4322-be4d-4ae64e424c53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:23.478647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89003-039a-4322-be4d-4ae64e424c53 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:23.479988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 545/1111] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:23.486855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:23.487268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:23.487419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:23.507608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:23.509464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:23.578887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 567/1112] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:23.605914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f89003-039a-4322-be4d-4ae64e424c53 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:23.735444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f89003-039a-4322-be4d-4ae64e424c53 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:23.773837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:23.774690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:23.774975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:23.775250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] PATH_INFO: `/v3/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:23.776817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] RBAC: Authorizing `identity:get_role(role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:23.778205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:23.779600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 546/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:23 2022] GET /identity/v3/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:23.785467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:23.878219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:23.885621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bc83c-3030-4fea-bed4-ae21a700b651 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:23.922401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:23.923270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:23.923560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:23.923837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] PATH_INFO: `/v3/users/e8a9be0cd0684722bfa2349ea2f3d492` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:23.937428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] RBAC: Authorizing `identity:get_user(user_id=e8a9be0cd0684722bfa2349ea2f3d492)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:23.938888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:23.940590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 568/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:23 2022] GET /identity/v3/users/e8a9be0cd0684722bfa2349ea2f3d492 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:23.947259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:24.013959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:24.061052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ddfaf-4de0-44f7-86d2-a0ab851b77f7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:24.097652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:24.098297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:24.098581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:24.098857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:24.164662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:24.166134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:24.234939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:24.270990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 547/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:24.276774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:24.380074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435eed2-a0fb-4006-a39c-26a8932910a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:24.412971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:24.413831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:24.414116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:24.414390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:24.415548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:24.417208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666178e-5d47-4cb0-9f40-61d08f713efc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:24.418900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 569/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:24 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:24.425007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:24.521876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7a27-a780-4daa-9363-07d3e2c5d16f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:24.555973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:24.556635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:24.556926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:24.557206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:24.558027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=3a496fa70bbf4c70819d5a7e212232c1, user.id=e8a9be0cd0684722bfa2349ea2f3d492) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:24.558340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:24.559777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b25d4f-8334-4b0f-bf93-d8b9339f2562 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:24.565099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 548/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:24 2022] GET /identity/v3/role_assignments?user.id=e8a9be0cd0684722bfa2349ea2f3d492&scope.project.id=3a496fa70bbf4c70819d5a7e212232c1&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:26.744860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ccf2f-259a-4fbb-b785-6b529480018c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:26.745421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ccf2f-259a-4fbb-b785-6b529480018c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:26.745783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ccf2f-259a-4fbb-b785-6b529480018c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:26.748148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 570/1118] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:26.753466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:26.753927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:26.754143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:26.771070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:26.772253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:26.795543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:26.837537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 549/1119] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:26.860283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ccf2f-259a-4fbb-b785-6b529480018c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:26.884989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:26.959181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ccf2f-259a-4fbb-b785-6b529480018c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:26.992446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:26.993306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:26.993592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:26.993864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] PATH_INFO: `/v3/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:26.995368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] RBAC: Authorizing `identity:get_role(role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:26.996688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:26.998037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 571/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:26 2022] GET /identity/v3/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:27.003854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:27.099435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20405e95-87a8-4c50-9acc-07aa336af2bf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:27.132006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:27.132889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:27.133171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:27.133442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] PATH_INFO: `/v3/users/e8a9be0cd0684722bfa2349ea2f3d492` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:27.134675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] RBAC: Authorizing `identity:get_user(user_id=e8a9be0cd0684722bfa2349ea2f3d492)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:27.136042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:27.137678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 550/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:27 2022] GET /identity/v3/users/e8a9be0cd0684722bfa2349ea2f3d492 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:27.143326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:27.230659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:27.240928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e761-aede-40da-9de4-17efe4f1c463 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:27.273739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:27.274512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:27.274793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:27.275065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:27.276217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:27.277664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3a344-e4a8-4660-adf6-94bce5d81a79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:27.279323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 572/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:27 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:27.285171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:27.312842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:27.382272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337427af-7136-412d-8c96-09e4b71ddc76 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:27.415344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db730-5f3d-453d-abad-be936048b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:27.416196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db730-5f3d-453d-abad-be936048b584 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:27.416474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db730-5f3d-453d-abad-be936048b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:27.416775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db730-5f3d-453d-abad-be936048b584 None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1/users/e8a9be0cd0684722bfa2349ea2f3d492/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:27.419057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559db730-5f3d-453d-abad-be936048b584 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a496fa70bbf4c70819d5a7e212232c1, user_id=e8a9be0cd0684722bfa2349ea2f3d492, role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:27.420490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559db730-5f3d-453d-abad-be936048b584 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:27.433367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 551/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:49:27 2022] PUT /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1/users/e8a9be0cd0684722bfa2349ea2f3d492/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:49:29.631153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa56216-d155-4686-9fe5-3e7a441f3a19 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:29.631681 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa56216-d155-4686-9fe5-3e7a441f3a19 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:29.631773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa56216-d155-4686-9fe5-3e7a441f3a19 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:29.633142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 573/1124] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:29.638221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:29.638591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:29.638771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:29.655560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:29.656766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:29.720058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 552/1125] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:29.743753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa56216-d155-4686-9fe5-3e7a441f3a19 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:29.843092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa56216-d155-4686-9fe5-3e7a441f3a19 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:29.875660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:29.876492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:29.876810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:29.877088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] PATH_INFO: `/v3/roles/2e4c51bd98ca4f7696808182d9c4871a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:29.879129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] RBAC: Authorizing `identity:get_role(role_id=2e4c51bd98ca4f7696808182d9c4871a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:29.880501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:29.881941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 574/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:29 2022] GET /identity/v3/roles/2e4c51bd98ca4f7696808182d9c4871a => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:29.887561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:29.906592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:29.986330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd93c4-e968-44f3-a565-eca780c65127 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:30.020090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:30.021490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:30.021779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:30.022052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] PATH_INFO: `/v3/users/e8a9be0cd0684722bfa2349ea2f3d492` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:30.023235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] RBAC: Authorizing `identity:get_user(user_id=e8a9be0cd0684722bfa2349ea2f3d492)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:30.024722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:30.026383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 553/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:29 2022] GET /identity/v3/users/e8a9be0cd0684722bfa2349ea2f3d492 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:30.032128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:30.058877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:30.130509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fab0d1-0d6d-465a-bd5d-af328c4800d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:30.163436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:30.164007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:30.164291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:30.164567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:30.230320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:30.231811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:30.258425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:30.330404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 575/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:30.336416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:30.419389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:30.434906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a435e2-4200-456b-9d15-476b55d90d85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:30.468560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:30.469435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:30.469721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:30.469998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] PATH_INFO: `/v3/projects/3a496fa70bbf4c70819d5a7e212232c1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:30.471171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] RBAC: Authorizing `identity:get_project(project_id=3a496fa70bbf4c70819d5a7e212232c1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:30.472684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e009ea-43d7-4374-985b-969b085b723a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:30.474364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 554/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:30 2022] GET /identity/v3/projects/3a496fa70bbf4c70819d5a7e212232c1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:30.480671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:30.577319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ee6e7-e5a1-47b7-a405-a815a32ba53d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:30.602321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:30.611420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:30.612059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:30.612345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:30.612633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:30.614066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] RBAC: Adding query filter params (role.id=2e4c51bd98ca4f7696808182d9c4871a, scope.project.id=3a496fa70bbf4c70819d5a7e212232c1, user.id=e8a9be0cd0684722bfa2349ea2f3d492) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:30.614376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:30.615795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31d9d5-75ec-40c3-aed9-9c49cc96b939 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:30.621536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 576/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:30 2022] GET /identity/v3/role_assignments?user.id=e8a9be0cd0684722bfa2349ea2f3d492&scope.project.id=3a496fa70bbf4c70819d5a7e212232c1&role.id=2e4c51bd98ca4f7696808182d9c4871a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:32.862606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b3856-4b2f-4b9f-bbe5-5427ff0e34fc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:32.863709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b3856-4b2f-4b9f-bbe5-5427ff0e34fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:32.863826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b3856-4b2f-4b9f-bbe5-5427ff0e34fc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:32.865176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 555/1131] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:32.870529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:32.870899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:32.871089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:32.887755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:32.888966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:32.953194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 577/1132] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:32.976500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b3856-4b2f-4b9f-bbe5-5427ff0e34fc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:33.076565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b3856-4b2f-4b9f-bbe5-5427ff0e34fc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:33.109623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:33.110398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:33.110683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:33.110958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:33.112121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:33.114281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e89b43-616c-4436-b1c2-afc3eb93c09a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:33.115837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 556/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:33.121576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:33.216388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c270e-2e61-4dfa-a3d8-129325724693 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:33.236185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:33.250064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:33.250709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:33.250990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:33.251264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:33.251964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:33.253453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ddc3-c22a-4aa5-80fb-c3669265a929 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:33.279939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 578/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:49:33 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:35.522130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645479-b68d-4ebe-84ef-1a49b3184219 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:35.522659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645479-b68d-4ebe-84ef-1a49b3184219 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:35.522659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9645479-b68d-4ebe-84ef-1a49b3184219 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:35.524052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 557/1135] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:35.529396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:35.529816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:35.530003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:35.546478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:35.547657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:35.614797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 579/1136] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:35.639589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9645479-b68d-4ebe-84ef-1a49b3184219 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:35.663321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:35.740292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9645479-b68d-4ebe-84ef-1a49b3184219 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:35.773622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:35.774341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:35.774628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:35.774940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:35.776115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:35.777583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2571d2-57db-4a21-84e4-878062615441 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:35.779173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 558/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:35.785349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:35.883196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0575b9-329d-4c9b-bd19-9db6703dcc56 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:35.884706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:35.917741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:35.918398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:35.918682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:35.918958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:35.919672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:35.921148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58602f-3d92-4ff7-838a-c842ba1245d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:35.940077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 580/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:49:35 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:38.200593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7aeab-1d21-457d-b6cb-61949ff53bc7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.201160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7aeab-1d21-457d-b6cb-61949ff53bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.201160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7aeab-1d21-457d-b6cb-61949ff53bc7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.202511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 559/1139] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:38.208686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.209087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.209250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.225740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:38.226939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:38.292513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 581/1140] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:38.316084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7aeab-1d21-457d-b6cb-61949ff53bc7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:38.404637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:38.415574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7aeab-1d21-457d-b6cb-61949ff53bc7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:38.447486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:38.448256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.448537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.448840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.459420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:38.466946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:38.468068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:38.469132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 560/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:38.474911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:38.558709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:38.570936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f55afe-47b9-42ae-b546-c442144682ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:38.603847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:38.604490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.604805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.605085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.605821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:38.606127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:38.607467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:38.615879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 582/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:38.621692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:38.717764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5cf11-f3be-4685-9588-d63d3656b671 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:38.752542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:38.754131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.754407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.754790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] PATH_INFO: `/v3/users/a6db874462fe4db5962da1e82c8a0857` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.768490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] RBAC: Authorizing `identity:get_user(user_id=a6db874462fe4db5962da1e82c8a0857)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:38.770005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:38.771675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 561/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:38 2022] GET /identity/v3/users/a6db874462fe4db5962da1e82c8a0857 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:38.777425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:38.878023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225145f5-2b34-4bd5-8382-c91ff6ae233e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:38.922184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:38.923126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:38.923553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:38.923972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:38.947237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:38.997673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:38.999114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:39.095198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 583/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:39.101667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:39.112343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:39.198315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eab0-5da3-48e4-b4ea-37a5491a4566 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:39.240434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:39.241339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:39.241619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:39.241893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] PATH_INFO: `/v3/projects/cfb64840d9664cbe9c69abc10257deb6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:39.243114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] RBAC: Authorizing `identity:get_project(project_id=cfb64840d9664cbe9c69abc10257deb6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:39.244724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57358e1b-be0a-4fa3-923f-fb3097385498 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:39.246620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 562/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:39 2022] GET /identity/v3/projects/cfb64840d9664cbe9c69abc10257deb6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:39.252903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:39.303784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:39.351207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d2de-7ffd-435a-971e-388237b058e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:39.384435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:39.385102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:39.385383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:39.385655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:39.386471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=cfb64840d9664cbe9c69abc10257deb6, user.id=a6db874462fe4db5962da1e82c8a0857) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:39.386779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:39.388206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5365-4ea7-436f-b149-238cc5a4a32b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:39.393502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 584/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:39 2022] GET /identity/v3/role_assignments?user.id=a6db874462fe4db5962da1e82c8a0857&scope.project.id=cfb64840d9664cbe9c69abc10257deb6&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:41.554096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b8aac-9a6f-4512-8964-d534aec0bf4b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:41.554648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b8aac-9a6f-4512-8964-d534aec0bf4b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:41.554648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b8aac-9a6f-4512-8964-d534aec0bf4b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:41.556113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 563/1147] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:41.561405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:41.561780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:41.561965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:41.578458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:41.579629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:41.645036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 585/1148] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:41.667712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b8aac-9a6f-4512-8964-d534aec0bf4b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:41.744557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:41.767865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b8aac-9a6f-4512-8964-d534aec0bf4b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:41.800422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:41.801213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:41.801494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:41.801765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:41.811554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:41.819488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:41.820565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:41.821470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 564/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:41.826812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:41.871152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:41.952029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455fbf2-e7d7-47c5-83e6-2043d5084987 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:41.984915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:41.985570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:41.985855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:41.986129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:41.986866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:41.987172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:41.988556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:41.996354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 586/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:42.002224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:42.100495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39d6c7-bbcd-4b4a-9979-02c36d450dd3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:42.133425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:42.134271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:42.134546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:42.134818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] PATH_INFO: `/v3/users/a6db874462fe4db5962da1e82c8a0857` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:42.135977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] RBAC: Authorizing `identity:get_user(user_id=a6db874462fe4db5962da1e82c8a0857)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:42.137430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:42.139138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 565/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:42 2022] GET /identity/v3/users/a6db874462fe4db5962da1e82c8a0857 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:42.145420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:42.247071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f0f7-cdfc-48e9-b9fc-a5e7358fe756 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:42.254810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:42.281609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:42.282448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:42.282731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:42.283005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] PATH_INFO: `/v3/projects/cfb64840d9664cbe9c69abc10257deb6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:42.284168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] RBAC: Authorizing `identity:get_project(project_id=cfb64840d9664cbe9c69abc10257deb6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:42.285627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0526b-be23-438a-aba9-3e687aba49fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:42.287293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 587/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:42 2022] GET /identity/v3/projects/cfb64840d9664cbe9c69abc10257deb6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:42.293247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:42.296995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:42.392020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6ce19-24f3-4368-b237-c31d06872f5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:42.424086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:42.425530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:42.425775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:42.426191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] PATH_INFO: `/v3/projects/cfb64840d9664cbe9c69abc10257deb6/users/a6db874462fe4db5962da1e82c8a0857/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:42.428138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] RBAC: Authorizing `identity:create_grant(project_id=cfb64840d9664cbe9c69abc10257deb6, user_id=a6db874462fe4db5962da1e82c8a0857, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:42.429616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9657dc-4e89-461e-8b4d-31246d17159a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:42.442662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 566/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:49:42 2022] PUT /identity/v3/projects/cfb64840d9664cbe9c69abc10257deb6/users/a6db874462fe4db5962da1e82c8a0857/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:49:44.638029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10d49d-ca53-498f-b0b7-c053c9c5aa5d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:44.638590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10d49d-ca53-498f-b0b7-c053c9c5aa5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:44.638590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10d49d-ca53-498f-b0b7-c053c9c5aa5d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:44.639949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 588/1154] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:44.645640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:44.646018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:44.646188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:44.662760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:44.663939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:44.721997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:44.732185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 567/1155] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:49:44.757396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10d49d-ca53-498f-b0b7-c053c9c5aa5d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:44.859267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10d49d-ca53-498f-b0b7-c053c9c5aa5d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:44.892223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:44.892962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:44.893251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:44.893532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:44.904076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:44.911836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:44.912981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:44.914020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 589/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:44.919145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:45.029767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490d9c9-38e6-4cfe-b7d5-b7db626fdd57 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:45.062108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:45.062767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:45.063047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:45.063322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:45.064058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:45.064362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:45.066326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:45.074062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 568/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:45.079831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:45.115076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:45.180323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558965aa-e39b-4051-9f44-83894a00f123 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:45.212384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:45.213251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:45.213534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:45.213808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] PATH_INFO: `/v3/users/a6db874462fe4db5962da1e82c8a0857` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:45.214975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] RBAC: Authorizing `identity:get_user(user_id=a6db874462fe4db5962da1e82c8a0857)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:45.216399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:45.218025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 590/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:45 2022] GET /identity/v3/users/a6db874462fe4db5962da1e82c8a0857 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:45.223778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:45.278405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:45.321028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd815d-e92e-4c23-8910-77e3b4819235 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:45.353140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:45.354036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:45.354319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:45.354591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:45.419221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:45.420690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:45.472393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:45.515562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 569/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:45.521634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:45.619625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaed94-0fd2-4c08-883e-37d9aa739d67 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:45.621143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:45.653030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:45.653880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:45.654167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:45.654447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] PATH_INFO: `/v3/projects/cfb64840d9664cbe9c69abc10257deb6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:45.655619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] RBAC: Authorizing `identity:get_project(project_id=cfb64840d9664cbe9c69abc10257deb6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:45.657083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb87b29-ee74-4e4b-a03f-6343a35ba69c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:45.658772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 591/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:45 2022] GET /identity/v3/projects/cfb64840d9664cbe9c69abc10257deb6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:45.665264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:45.761022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c443-0444-47e7-85a5-a130e18bbc12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:45.793543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:45.794185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:45.794472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:45.794753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:45.795575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=cfb64840d9664cbe9c69abc10257deb6, user.id=a6db874462fe4db5962da1e82c8a0857) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:45.795888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:45.797347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ffa7f-2fc3-4559-8d10-ad58926d9835 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:45.803114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 570/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:45 2022] GET /identity/v3/role_assignments?user.id=a6db874462fe4db5962da1e82c8a0857&scope.project.id=cfb64840d9664cbe9c69abc10257deb6&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:48.073843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5b9c3-ea36-435d-ab27-293af094c977 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:48.074397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5b9c3-ea36-435d-ab27-293af094c977 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:48.074397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5b9c3-ea36-435d-ab27-293af094c977 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:48.075754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 592/1162] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:48.080873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:48.081315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:48.081710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:48.098774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:48.099952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:48.114540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:48.166541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 571/1163] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:48.188007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5b9c3-ea36-435d-ab27-293af094c977 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:48.287730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5b9c3-ea36-435d-ab27-293af094c977 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:48.321025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:48.321665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:48.321953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:48.322233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:48.360676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:48.388381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:48.389926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:48.485622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 593/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:48.492307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:48.587889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df307807-0dd3-4623-b032-2239bf46cadf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:48.621170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:48.621948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:48.622233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:48.622512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:48.629637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:48.631093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3f4c4ac3-b641-40c7-a999-30966e722cee None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:48.645260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:48.646516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:49:48.647449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 572/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:49:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:48.652214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:48.717488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:48.748406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a296ab8-29cb-414f-b5eb-7988072c8e77 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:48.781655 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:48.782300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:48.782581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:48.782854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:48.783563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:48.783867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:48.786336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efefba-9f23-43f3-80f0-056a912f2d61 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:48.793343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 594/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Thu May 19 21:49:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:51.007789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62121fa9-c2b7-4951-8425-e8f9b1479651 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:51.008339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62121fa9-c2b7-4951-8425-e8f9b1479651 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:51.008339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62121fa9-c2b7-4951-8425-e8f9b1479651 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:51.009762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 573/1167] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:51.015107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417dc73-aec2-4c71-bd93-ace905eb9d61 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:51.015526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417dc73-aec2-4c71-bd93-ace905eb9d61 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:51.015659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417dc73-aec2-4c71-bd93-ace905eb9d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:51.033428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f417dc73-aec2-4c71-bd93-ace905eb9d61 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:51.034653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f417dc73-aec2-4c71-bd93-ace905eb9d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:51.099822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 595/1168] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:51.122490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62121fa9-c2b7-4951-8425-e8f9b1479651 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:51.135753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:51.222858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62121fa9-c2b7-4951-8425-e8f9b1479651 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:51.256228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:51.256903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:51.257188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:51.257465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:51.258243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:51.259579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4292-2247-4114-b3e0-78aa4a16e1ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:51.274086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 574/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Thu May 19 21:49:51 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:53.502520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ceb3-aa9f-4269-a922-5f09e072c6a9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:53.503224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ceb3-aa9f-4269-a922-5f09e072c6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:53.503224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ceb3-aa9f-4269-a922-5f09e072c6a9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:53.504517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 596/1170] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:53.509913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:53.510311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:53.510482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:53.527692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:53.528913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:53.590834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:53.597091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 575/1171] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:53.619548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56ceb3-aa9f-4269-a922-5f09e072c6a9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:53.682011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:53.720031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56ceb3-aa9f-4269-a922-5f09e072c6a9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:53.753899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:53.754682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:53.754966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:53.755246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] PATH_INFO: `/v3/domains/d38684475b754f64957957b4bbe4b049` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:53.765370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d38684475b754f64957957b4bbe4b049)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:53.766792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e84a9b-b5da-43b0-bf80-58e8241c50d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:53.768492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 597/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Thu May 19 21:49:53 2022] GET /identity/v3/domains/d38684475b754f64957957b4bbe4b049 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:53.774258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:53.873401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dadaa-f3b9-4f62-b884-ed3ebaf3299f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:53.906606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:53.907255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:53.907539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:53.907818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:53.908532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:53.911006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7d273-ce9b-457a-956f-184ab54eb5b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:53.936139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 576/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:49:53 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:56.184569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c7545-dffd-42f3-a36f-49df884eaa9b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:56.185141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c7545-dffd-42f3-a36f-49df884eaa9b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:56.185141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c7545-dffd-42f3-a36f-49df884eaa9b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:56.186521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 598/1174] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:56.192410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:56.192858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:56.193011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:56.210066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:56.211243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:56.257204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:56.277929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 577/1175] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:56.302498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c7545-dffd-42f3-a36f-49df884eaa9b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:56.400626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c7545-dffd-42f3-a36f-49df884eaa9b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:56.422971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:56.434228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:56.435065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:56.435345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:56.435618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] PATH_INFO: `/v3/domains/d38684475b754f64957957b4bbe4b049` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:56.436792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d38684475b754f64957957b4bbe4b049)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:56.438206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05597929-86b7-4a2d-a240-0caa46732f09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:56.439784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 599/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Thu May 19 21:49:56 2022] GET /identity/v3/domains/d38684475b754f64957957b4bbe4b049 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:56.446160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:56.540535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410abc2-3106-41bc-91c0-dc218f62cfa0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:56.573730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:56.574327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:56.574614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:56.574897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:56.575607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:56.577082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99db0a-d11b-49aa-bdfc-436a1f1f4511 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:56.596187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 578/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:49:56 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:49:58.839095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc04a-1ae5-4d44-b3c8-ba0faf644807 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:58.839653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc04a-1ae5-4d44-b3c8-ba0faf644807 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:58.839653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc04a-1ae5-4d44-b3c8-ba0faf644807 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:58.841066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 600/1178] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:49:58.846324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:58.846751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:58.846888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:58.865261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:58.866437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:58.896636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:58.934752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 579/1179] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:49:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:49:58.964135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc04a-1ae5-4d44-b3c8-ba0faf644807 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.066254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc04a-1ae5-4d44-b3c8-ba0faf644807 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:59.099741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:59.100456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:59.100774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:59.101056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:59.111093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:49:59.118670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:49:59.119755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:49:59.121015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 601/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:49:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:49:59.125822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.221241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654872-28f8-4cb2-98aa-590d80a2c48c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:59.254306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:59.254952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:59.255233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:59.255504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:59.256237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:49:59.256537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:59.257929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:59.265664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 580/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:49:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:59.271837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.325291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:59.372664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd98ef-676a-4043-975f-70e70fafd5b1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:59.405842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:59.406701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:59.406984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:59.407260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] PATH_INFO: `/v3/users/799ff79cfffe4f35a97ffd2c944efeca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:59.420473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] RBAC: Authorizing `identity:get_user(user_id=799ff79cfffe4f35a97ffd2c944efeca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:59.421932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:59.423554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 602/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:49:59 2022] GET /identity/v3/users/799ff79cfffe4f35a97ffd2c944efeca => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:49:59.429944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.460455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:59.528929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e683bc-3307-4f67-b1bb-b18cbc4f7fb2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:59.561598 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:59.562232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:59.562568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:59.562852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:59.628678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:59.630089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:59.658044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:59.725533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 581/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:49:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:49:59.731559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.845934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea257e-2c21-4f04-9c15-792e6bc151c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:59.857778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:59.882007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:59.882798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:59.883087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:59.883368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] PATH_INFO: `/v3/projects/167a3025168840db963f7697f9874e03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:59.884543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] RBAC: Authorizing `identity:get_project(project_id=167a3025168840db963f7697f9874e03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:59.886010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025719f-04df-4420-b01f-ed17b249279b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:59.887700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 603/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:49:59 2022] GET /identity/v3/projects/167a3025168840db963f7697f9874e03 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:49:59.893799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:59.990229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9cc01-31d3-4e98-bf09-70a17fb0bcbc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:00.017609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:00.023800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:00.024447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:00.024931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:00.025077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:00.025899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=167a3025168840db963f7697f9874e03, user.id=799ff79cfffe4f35a97ffd2c944efeca) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:50:00.026213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:00.027672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5240-c290-4fd5-acd4-35620cdc71ff None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:00.033249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 582/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:49:59 2022] GET /identity/v3/role_assignments?user.id=799ff79cfffe4f35a97ffd2c944efeca&scope.project.id=167a3025168840db963f7697f9874e03&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:02.220600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8a6-8098-4056-b5c5-f376560e5733 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.221689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8a6-8098-4056-b5c5-f376560e5733 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.221789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac8a6-8098-4056-b5c5-f376560e5733 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.223131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 604/1186] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:50:02.228439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.228847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.229022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.245359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:50:02.246527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:50:02.310148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 583/1187] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:50:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:50:02.332625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac8a6-8098-4056-b5c5-f376560e5733 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:02.432165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac8a6-8098-4056-b5c5-f376560e5733 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:02.464606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:02.465404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.465689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.465963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.475953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:50:02.483704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:50:02.484844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:50:02.485855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 605/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:50:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:50:02.491251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:02.591927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b393a-4ce2-43bf-a2e3-c95408d602e3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:02.614922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:02.625832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:02.626487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.626775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.627052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.628457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:50:02.628793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:02.630183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:02.638135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 584/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:02.643877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:02.683246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:02.742858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44f783-e987-4496-aadd-bf5591702b62 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:02.775822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:02.776695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.776987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.777270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] PATH_INFO: `/v3/users/799ff79cfffe4f35a97ffd2c944efeca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.778449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] RBAC: Authorizing `identity:get_user(user_id=799ff79cfffe4f35a97ffd2c944efeca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:02.779902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:02.781545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 606/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:50:02 2022] GET /identity/v3/users/799ff79cfffe4f35a97ffd2c944efeca => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:02.787762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:02.891179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299ae31-4d35-43af-864c-ffaa24e1f00a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:02.925475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:02.926323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:02.926609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:02.926887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] PATH_INFO: `/v3/projects/167a3025168840db963f7697f9874e03` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:02.928061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] RBAC: Authorizing `identity:get_project(project_id=167a3025168840db963f7697f9874e03)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:02.929531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64be6fa-7ca9-4dc3-9e48-4e47e04aeec7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:02.931223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 585/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:50:02 2022] GET /identity/v3/projects/167a3025168840db963f7697f9874e03 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:02.937335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:03.038372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b1e2d-8290-4ca4-810f-49dea442c188 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:03.045592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:03.073635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:03.074513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:03.074797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:03.075071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] PATH_INFO: `/v3/projects/167a3025168840db963f7697f9874e03/users/799ff79cfffe4f35a97ffd2c944efeca/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:03.077036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] RBAC: Authorizing `identity:create_grant(project_id=167a3025168840db963f7697f9874e03, user_id=799ff79cfffe4f35a97ffd2c944efeca, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:03.078458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa328af-880a-4a59-bea7-72a64f362570 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:03.091352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 607/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:50:02 2022] PUT /identity/v3/projects/167a3025168840db963f7697f9874e03/users/799ff79cfffe4f35a97ffd2c944efeca/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:50:05.334438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7343512-e6c0-44c3-a0b5-540e3f1058d5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:05.334979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7343512-e6c0-44c3-a0b5-540e3f1058d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:05.334979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7343512-e6c0-44c3-a0b5-540e3f1058d5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:05.336360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 586/1193] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:50:05.341767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55a91-a18c-4680-a792-77666542f182 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:05.342191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55a91-a18c-4680-a792-77666542f182 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:05.342330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55a91-a18c-4680-a792-77666542f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:05.359132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14c55a91-a18c-4680-a792-77666542f182 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:50:05.360306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14c55a91-a18c-4680-a792-77666542f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:50:05.425921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 608/1194] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:50:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:50:05.449449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7343512-e6c0-44c3-a0b5-540e3f1058d5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:05.505590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:05.559187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7343512-e6c0-44c3-a0b5-540e3f1058d5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:05.591305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:05.591993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:05.592272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:05.592545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:05.602813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:50:05.610505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:50:05.612121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:50:05.617556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 587/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:50:05.618417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c55a91-a18c-4680-a792-77666542f182 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:05.710085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:05.716175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c55a91-a18c-4680-a792-77666542f182 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:05.750242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:05.750901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:05.751185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:05.751462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:05.752783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:50:05.753234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:05.754604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:05.762675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 609/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:05.768526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:05.866064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef951-f234-4eb4-a56b-5f1a7585d090 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:05.879659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:05.899291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:05.900148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:05.900434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:05.900741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] PATH_INFO: `/v3/users/799ff79cfffe4f35a97ffd2c944efeca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:05.901926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] RBAC: Authorizing `identity:get_user(user_id=799ff79cfffe4f35a97ffd2c944efeca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:05.903372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:05.905013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 588/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Thu May 19 21:50:05 2022] GET /identity/v3/users/799ff79cfffe4f35a97ffd2c944efeca => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:05.911272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:06.010263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd3acf-4584-4136-94bb-47bd319b4da4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:06.044368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:06.045038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:06.045319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:06.045591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:06.072702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:06.113952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:06.115415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:06.210074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 610/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:50:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:50:06.215848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:06.312191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb72f54-0086-4bb1-94a3-0265dfbb8cdc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:06.345425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:06.346880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:06.347169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:06.347450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] PATH_INFO: `/v3/projects/167a3025168840db963f7697f9874e03` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:06.348656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] RBAC: Authorizing `identity:get_project(project_id=167a3025168840db963f7697f9874e03)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:06.350111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabf583-2b89-4e92-b29c-64f978cbc1df None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:06.351789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 589/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Thu May 19 21:50:06 2022] GET /identity/v3/projects/167a3025168840db963f7697f9874e03 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:06.357970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:50:06.414249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:06.454234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4974ed-160b-4617-8e91-05d541683072 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:50:06.488018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:50:06.488597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:50:06.488911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:50:06.489185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:50:06.490005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=167a3025168840db963f7697f9874e03, user.id=799ff79cfffe4f35a97ffd2c944efeca) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:50:06.490314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:50:06.491715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81d3e-f56b-4b40-865b-fb827842d5ae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:50:06.497447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 611/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:50:06 2022] GET /identity/v3/role_assignments?user.id=799ff79cfffe4f35a97ffd2c944efeca&scope.project.id=167a3025168840db963f7697f9874e03&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:50:16.341735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=75 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:50:16.494874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=39 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:07.337788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f35b07-ccbc-411d-8050-e0ca94528eb9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:07.338370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f35b07-ccbc-411d-8050-e0ca94528eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:07.338468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f35b07-ccbc-411d-8050-e0ca94528eb9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:07.339801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 590/1201] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:07.345836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:07.346336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:07.346424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:07.369811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:07.371024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:07.463195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 612/1202] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:07.487954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f35b07-ccbc-411d-8050-e0ca94528eb9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:07.632830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f35b07-ccbc-411d-8050-e0ca94528eb9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:07.679659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:07.680477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:07.680800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:07.681079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:07.696779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:07.698822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:07.732583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:07.734029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:07.736072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:07.736072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:07.736072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:07.736072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:07.736072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 591/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:07.742451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:07.885435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e957870-5baa-4adc-b7e4-0cc007cef82e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:07.923699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:07.933257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:07.933988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:07.934287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:07.934584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:07.935400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:07.935813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:07.937294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e018d-bdb3-40f9-b599-bbc48074de59 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:07.952672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 613/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:07.956977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:08.021181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:08.089388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9f6d-f554-4d22-9da7-c9363b35c61f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:08.135777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:08.136553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:08.136888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:08.137223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:08.137950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:08.139632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f221d38-18b0-4ada-924a-8183af247340 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:08.168792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 592/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Thu May 19 21:51:07 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:10.730799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304514-9289-4d1c-a78b-25f1ce66f70a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:10.730799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304514-9289-4d1c-a78b-25f1ce66f70a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:10.730799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304514-9289-4d1c-a78b-25f1ce66f70a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:10.730799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 614/1206] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:10.736812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:10.737274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:10.737365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:10.758814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:10.760242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:10.836707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 593/1207] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:10.861005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304514-9289-4d1c-a78b-25f1ce66f70a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:10.976473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304514-9289-4d1c-a78b-25f1ce66f70a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.014599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.016722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.016722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.034675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:11.042736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:11.043679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:11.044527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 615/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:51:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:11.050202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:11.092104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:11.161731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cb2d9-5920-4d26-94ad-e472cf5b8328 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.198493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.199164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.199455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.199763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.200476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:11.200812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:11.202194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:11.211573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 594/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:11.217372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:11.278805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:11.337803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff699-2c13-434b-b42b-ad26392c2c7d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.379237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.380063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.380392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.380732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.394913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:11.394913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:11.414094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:11.415570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:11.416739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:11.416739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:11.416739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:11.416739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:11.416739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 616/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:11.421115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:11.532839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:11.539527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eddbe-f44d-4db9-b81f-ccb0e93e9589 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.580846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.581463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.581742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.582017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.582738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:11.583726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:11.584420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:11.599367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 595/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:11.605536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:11.736012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf58-d1e9-4624-bf63-14cce583cc35 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.738502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:11.783177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.783956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.784250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.784572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.793559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:11.795272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:11.814536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:11.815799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:11.817134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 617/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:11.821828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:11.947246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27206ba-d998-4653-81b8-b7cd4c1d96a6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:11.987575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:11.988301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:11.988589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:11.988899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:11.989907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:11.990268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:11.991846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.011929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 596/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:12.018235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:12.141730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c174d7-d46a-4313-bbf8-8d89b51177d0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:12.180943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:12.181636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:12.181962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:12.182268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:12.220480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:12.266915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:12.268572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.386168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 618/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:51:12.393989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:12.415460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:12.512954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3818d3-04be-450c-8fa4-c788fe8653f9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:12.559976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:12.560991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:12.561385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:12.561574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:12.574769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:12.576241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:12.595751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:12.597001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:12.598094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:12.598094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:12.598094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:12.598094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:12.602274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 597/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:12.603685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:12.681687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:12.726296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c854-43a0-4cdd-b1a5-35f8a71de0ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:12.764841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:12.767361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:12.767361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:12.767361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:12.767361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:12.767361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:12.768491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.780943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 619/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:12.786701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:12.868204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:12.911787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f0a1-c6e2-45a1-8f8e-e27ed5165589 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:12.952430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:12.953901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:12.954218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:12.954573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:12.961512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:12.963311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:12.982091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:12.983367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:12.985437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:12.989594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 598/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:12.990536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:13.131754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fed1-38b0-43c2-8360-e760ece4b0a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:13.157702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:13.175315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:13.176259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:13.176743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:13.177174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:13.178120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:13.178592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:13.180157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc0f0-7ca4-47fd-b15e-c66bf068fca3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:13.193874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 620/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:13.200244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:13.328716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689f3d-858f-430b-b0c6-1224ceb1f79e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:13.360869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:13.369628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:13.370668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:13.370770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:13.370966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:13.371892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=fb31582d6fc14a7f9ec20d107c445390) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:13.372225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:13.373832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ddaa3-9a42-4216-97b1-d104f2b6d69d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:13.380548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 599/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:51:13 2022] GET /identity/v3/role_assignments?user.id=fb31582d6fc14a7f9ec20d107c445390&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:15.821461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45406e3-b687-4113-a3ac-7869e0574aec None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.822035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45406e3-b687-4113-a3ac-7869e0574aec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.822035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45406e3-b687-4113-a3ac-7869e0574aec None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.823395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 621/1220] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:15.830124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.830864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.830864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.848762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:15.850255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:15.932171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 600/1221] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:15.955194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45406e3-b687-4113-a3ac-7869e0574aec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.072771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45406e3-b687-4113-a3ac-7869e0574aec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.109734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.110524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.110803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.111076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.121127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:16.130154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:16.131196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:16.132255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 622/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:16.137757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.239500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d69f-4f1b-4034-ba53-71d0ecb3e683 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.268710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:16.282328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.283307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.283643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.283919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.284720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:16.285052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.286539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.295290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 601/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:51:16.301322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.353389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:16.417792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3bf8e-3028-49d9-b135-8ba0b7a8bf39 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.457194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.458067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.458397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.458688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.471512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.473009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:16.490740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:16.491863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:16.493111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:16.493111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:16.493111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:16.493111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:16.493111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 623/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:16.497878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.604031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d623ad-6e9f-4e1f-a3ee-4cfb3280f2fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.640382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.641326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.641651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.641892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.642697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:16.643006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.644393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.657882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 602/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:16.663557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.773993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:16.783176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e7e0-c4c0-4795-8fd1-3887d6e9157f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.819605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.820407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.820732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.821013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.828822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.830240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:16.845005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:16.846033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:16.846967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:16.852387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 624/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:16.852843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.903009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:16.959444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f70c9-5f36-460a-9a64-09542a192ac6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.996952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.997628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.997949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.998213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.999175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:16.999577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.001209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.019695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 603/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:17.026331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:17.138464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9dcc-9ee4-4f2b-b8e5-a2bdd003acca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:17.177003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:17.177774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:17.178056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:17.178352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:17.189320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.191006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:17.209969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:17.211094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:17.216562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:17.216562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:17.216562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:17.216562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:17.216562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 625/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:17.216869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:17.330181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32fdae-fecf-4460-9a08-32473ad6a255 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:17.332686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:17.368829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:17.369596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:17.369901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:17.370208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:17.371055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:17.371393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.372929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.386083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 604/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:17.391986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:17.401693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:17.506451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcbe1d-5bac-484a-a073-904fd6cce7f7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:17.543114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:17.543943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:17.544236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:17.544515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:17.551957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.553634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:17.569090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:17.570209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:17.571453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 626/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:17.576157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:17.686502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c863-1fe7-416b-822c-ba50b9f7cd7d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:17.721991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:17.722709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:17.723021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:17.723331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:17.724219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:17.724546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.726084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf31848-d218-49e2-adbc-c3d75bc22e5f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.739960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 605/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:17.746173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:17.822246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:17.860209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62766f-3b08-4c9d-841b-d34d9adcfd75 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:17.908807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:17.909821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:17.910110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:17.910392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/fb31582d6fc14a7f9ec20d107c445390/roles/741fb1aea82f49f0b4d3115dcb22d80a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:17.931672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=fb31582d6fc14a7f9ec20d107c445390, role_id=741fb1aea82f49f0b4d3115dcb22d80a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:17.933199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579e147-3b4d-4c15-bbd7-2f1278a6099d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:17.946965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 627/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:51:17 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/fb31582d6fc14a7f9ec20d107c445390/roles/741fb1aea82f49f0b4d3115dcb22d80a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:20.195886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74cdb9-e099-4b2b-ae99-8d784cf25142 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:20.196727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74cdb9-e099-4b2b-ae99-8d784cf25142 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:20.196727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74cdb9-e099-4b2b-ae99-8d784cf25142 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:20.197866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 606/1233] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:20.205079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:20.205528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:20.205953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:20.224172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:20.225626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:20.301824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 628/1234] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:51:20.326569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74cdb9-e099-4b2b-ae99-8d784cf25142 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:20.354746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:20.442277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74cdb9-e099-4b2b-ae99-8d784cf25142 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:20.479872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:20.480709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:20.481001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:20.481278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:20.491977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:20.500248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:20.501612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:20.502825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 607/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Thu May 19 21:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:20.507921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:20.559598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:20.621129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4010bc5-570b-409d-bdb1-78331d71641b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:20.659274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:20.659919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:20.660207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:20.660486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:20.661263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:20.661574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:20.663020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:20.676052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 629/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Thu May 19 21:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:20.677847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:20.795053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:20.818930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdf1a6-6931-42be-ad08-02b11b8b5974 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:20.857723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:20.859131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:20.859505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:20.859942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:20.872908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:20.875210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:20.893537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:20.894697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:20.895795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:20.895795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:20.895795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:20.895795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:20.897384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 608/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:51:20.903234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:21.023386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851caa8-f56d-47d2-ab00-6d40f713099e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:21.034931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:21.064243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:21.064975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:21.065264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:21.065552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:21.066338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:21.066716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:21.068095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:21.080887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 630/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:51:21.087397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:21.197806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0995-0787-4311-ba37-5c531ff86940 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:21.235397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:21.236204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:21.236491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:21.236895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:21.239788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:21.245437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:21.247403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:21.263360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:21.264636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:21.267059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 609/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:21.272683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:21.384463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7ced-eed0-4e95-9a52-0e4ea2287203 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:21.424824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:21.425501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:21.425788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:21.426067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:21.426914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:21.427226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:21.428893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:21.453614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 631/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:21.459514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:21.567325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728b69-0652-4510-89d5-a7df4425b582 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:21.609062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:21.609726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:21.610089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:21.610362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:21.683353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:21.691724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:21.693309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:21.821556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 610/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:51:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:51:21.828113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:21.861456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:21.951903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc227-e395-4da3-84ad-92056d782c01 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:21.993279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:21.994107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:21.994400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:21.994700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:22.005786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:22.007529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:22.030976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:22.032592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:22.033541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:22.033541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:22.033541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:22.033541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:22.033541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 632/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:22.035718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:22.141871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:22.148931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8d9e-7ffb-40bf-86e1-72634a2aff46 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:22.189056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:22.189779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:22.190078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:22.190383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:22.191399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:22.191713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:22.193276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:22.207049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 611/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:22.212900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:22.302656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:22.327752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468b37f-21df-4571-baaa-ec3b7029d5e7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:22.365730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:22.366615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:22.366929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:22.367245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:22.375735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:22.377377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:22.391757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:22.392929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:22.398661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 633/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:22.399270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:22.511068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c520f2-963e-4c50-8db6-b32a49a8b66d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:22.548960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:22.549691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:22.549993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:22.550286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:22.551115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:22.551484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:22.553458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375ff59-24bc-4c8e-a63b-ab1e5b1af26c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:22.567279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 612/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:22.573509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:22.691449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41af3e-5edd-4c4d-8029-8d6b70cd4262 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:22.730078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:22.730818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:22.731105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:22.731383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:22.732215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] RBAC: Adding query filter params (role.id=741fb1aea82f49f0b4d3115dcb22d80a, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=fb31582d6fc14a7f9ec20d107c445390) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:22.732533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:22.734184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9b0f-b0b7-4394-853a-2177f17e0611 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:22.740724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 634/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:51:22 2022] GET /identity/v3/role_assignments?user.id=fb31582d6fc14a7f9ec20d107c445390&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=741fb1aea82f49f0b4d3115dcb22d80a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:25.189597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716a880-a1f7-4622-af90-bbbcf4aa30ca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.190329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716a880-a1f7-4622-af90-bbbcf4aa30ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.190329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716a880-a1f7-4622-af90-bbbcf4aa30ca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.191692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 613/1247] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:25.197327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.197829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.197939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.204468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:25.220649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:25.222212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.301364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 635/1248] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:25.325957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716a880-a1f7-4622-af90-bbbcf4aa30ca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.348668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:25.445615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716a880-a1f7-4622-af90-bbbcf4aa30ca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.488556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.489456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.489740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.490020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.504099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:25.512424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:25.513596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:25.514592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 614/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:25.520038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.631317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa4c7-66d6-4696-a5ff-5bddf21681b2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.667717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:25.673814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.674490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.674778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.675334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.676100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:25.676419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:25.677844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:25.686602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 636/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:51:25.692826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.796935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:25.825013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604781-8641-4f6a-9261-a6f9c2fe3b4a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.867215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.868106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.868398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.868712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.881138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:25.882648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:25.901895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:25.902893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:25.903809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:25.903809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:25.903809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:25.903809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:25.903809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 615/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:25.911014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.035614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dada2-be01-4805-85d2-0fbbcd4f9c23 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.074210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.074875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.075202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.075550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.076352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:26.076668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.078616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.090646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 637/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:26.097956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.209021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc517cc-a385-48fd-b167-882a237cd446 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.237265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.246368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.247168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.247454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.247732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.255952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.257460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:26.273234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:26.274675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:26.275978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 616/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:26.281051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.344166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.396212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5547-2ef3-4f5c-9e15-639661674a09 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.436967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785148b-c79e-4316-960d-60790102baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.437899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785148b-c79e-4316-960d-60790102baeb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.438188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785148b-c79e-4316-960d-60790102baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.438492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785148b-c79e-4316-960d-60790102baeb None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.439589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785148b-c79e-4316-960d-60790102baeb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:26.439975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785148b-c79e-4316-960d-60790102baeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.441814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785148b-c79e-4316-960d-60790102baeb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.463278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 638/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:26.469477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.587464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694be7-f59d-4fb8-a99c-b2790a4b814f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.628882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.629749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.630079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.630470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.698268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.713078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.714631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.833041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 617/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:51:26.838971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785148b-c79e-4316-960d-60790102baeb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.943030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785148b-c79e-4316-960d-60790102baeb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.958039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.982098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.982934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.983344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.983765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.995218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.996719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:27.015738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:27.016932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:27.017844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:27.017844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:27.017844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:27.017844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:27.017844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 639/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:27.024090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.119026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:27.139230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd380e2e-0ce0-4897-8cdd-018fb787358b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.176447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.177131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.177419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.177693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.178417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:27.178716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.180097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.192015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 618/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:27.197883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.299757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9046f9-af35-47f2-af25-dca8eba7f341 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.335113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.335903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.336186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.336462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.343256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.344712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:27.358602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:27.359676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:27.361098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 640/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:27.365675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.464749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fd923-6fc4-4ec9-aa2e-cb587adf158f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.497882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.498536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.498877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.499152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.500076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:27.500387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.501875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde3232-0926-4e85-8ad0-1306d8a01f35 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.513458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 619/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:51:27.519901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.545503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:27.619157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34c1b8-de91-44df-9adb-44afcbd4d15d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.651736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.652376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.652685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.652966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.653794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=fb31582d6fc14a7f9ec20d107c445390) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:27.654105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.656133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2fcfa-ddb1-484a-bbc9-8dc94eeed53b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.661616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 641/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/role_assignments?user.id=fb31582d6fc14a7f9ec20d107c445390&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:29.872336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33654d75-8d91-458b-a6c0-86608b24a628 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:29.872922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33654d75-8d91-458b-a6c0-86608b24a628 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:29.872922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33654d75-8d91-458b-a6c0-86608b24a628 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:29.874987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 620/1261] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:29.880076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:29.880445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:29.880663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:29.897790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:29.898962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:29.962298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 642/1262] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:29.984789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33654d75-8d91-458b-a6c0-86608b24a628 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:29.995547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.095370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33654d75-8d91-458b-a6c0-86608b24a628 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.129505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.130244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.130513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.130790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.140703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:30.148269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:30.149383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:30.150630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 621/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:30.155900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:30.174103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.252738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177eb2c-33f7-418c-851b-9aa418436ab1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.286274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.286920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.287201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.287474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.288204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:30.288505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:30.289886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:30.298309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 643/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:30.304044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:30.369560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.402191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341caab-90e1-4613-957a-5b55ffc7b7f2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.435295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.436069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.436482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.436806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.447742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:30.449252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:30.465845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:30.466895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:30.468382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:30.468382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:30.468382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:30.468382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:30.468382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 622/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:30.473171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:30.552906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.571609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85158828-17bd-4e29-ab24-618144d5e6dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.605067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.605723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.606012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.606291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.607010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:30.607318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:30.608794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:30.620215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 644/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:51:30.625789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:30.722496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dbd8e-f32e-4b6c-b532-29d7aa724c58 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.741342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.758873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.759853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.760188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.760478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.770486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:30.772281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:30.791833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:30.792974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:30.794196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 623/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:30.799315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:30.903275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce921c-fa4d-4822-be03-7e7c24423485 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:30.936808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:30.937606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:30.937895 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:30.938178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:30.939014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:30.939328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:30.940780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:30.949260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:30.958237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 645/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:30.963879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:31.063756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d1fd9-1cbd-42c6-8dc8-e01f75410c0a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:31.096519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:31.097333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:31.097617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:31.097894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:31.108593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:31.110075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:31.127175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:31.128101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:31.129241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:31.129241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:31.129241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:31.129241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:31.129241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 624/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:31.133942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:31.229298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae9cce-cb3b-475b-ab4f-25ec494b7bce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:31.261871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:31.262513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:31.262793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:31.263066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:31.263777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:31.264082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:31.266107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:31.276935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 646/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:31.282555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:31.314225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:31.381634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376525ce-5528-4207-b5b1-d4ec7a03914f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:31.413576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:31.414355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:31.414640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:31.414918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:31.421928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:31.423400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:31.437962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:31.438832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:31.439748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 625/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:31.444804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:31.502474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:31.542719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b36738-7261-4afd-9b0f-2bab6adac76e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:31.577368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:31.578108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:31.578392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:31.578668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:31.579451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:31.579760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:31.581225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3055e0b-f793-46f7-80c7-257f6d5b3512 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:31.593084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 647/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:31.598860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:31.688366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:31.697327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305ed5-3025-48e4-8cc7-5ce39960796d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:31.730132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:31.730999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:31.731285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:31.731562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] PATH_INFO: `/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/fb31582d6fc14a7f9ec20d107c445390/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:31.734128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] RBAC: Authorizing `identity:create_grant(project_id=df7a31e797b24706a0b85db1ee03cc75, user_id=fb31582d6fc14a7f9ec20d107c445390, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:31.735580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5e00f-d905-4b2e-9da0-dd5cf6044927 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:31.750477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 626/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Thu May 19 21:51:31 2022] PUT /identity/v3/projects/df7a31e797b24706a0b85db1ee03cc75/users/fb31582d6fc14a7f9ec20d107c445390/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:34.015252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010f22-0eda-46d9-8069-f997766f9a39 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.015816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010f22-0eda-46d9-8069-f997766f9a39 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.015816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010f22-0eda-46d9-8069-f997766f9a39 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.017234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 648/1274] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:34.022726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.023138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.023285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.040414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:34.041994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:34.109728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 627/1275] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:34.133344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46010f22-0eda-46d9-8069-f997766f9a39 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.241651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46010f22-0eda-46d9-8069-f997766f9a39 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:34.258859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:34.276056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:34.276810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.277100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.277379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.287267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:51:34.294807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:34.296388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:34.297448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 649/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:34.302717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.381465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:34.400116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa724ab5-832c-43ee-ad65-30a49b32838e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:34.433076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:34.433727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.434011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.434286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.435024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:34.435381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:34.437462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:34.445274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 628/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:34.451002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.550022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc63e26-0723-43ec-b817-d6fa1f15ff56 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:34.583184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:34.583953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.584243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.584524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.595765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:34.597260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:34.614685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:34.615816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:34.616943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:34.616943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:34.616943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:34.616943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:34.616943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 650/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:34.621630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.718583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b34ce-76ff-4ba0-9213-2af4c2c8159e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:34.752882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:34.753548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.753826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.754110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.754917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:34.755152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:34.756536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:34.769343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 629/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:34.775176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.808860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:34.877228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed7db-712b-44f8-a766-f1bd736543f0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:34.910225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:34.911013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:34.911298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:34.911571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:34.919750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:34.921742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:51:34.935770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:34.937019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:34.938179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 651/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:51:34.942964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:34.946665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.042767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a29ea7-6b97-4fc4-bb9a-2adacacf94e2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:35.074990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:35.075588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:35.075868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:35.076141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:35.077630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:35.077932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:35.079363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:35.096192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 630/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Thu May 19 21:51:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:35.101836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:35.190594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.200232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179b5c-754b-475a-8707-9df10a80eb2b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:35.234061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:35.234704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:35.234992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:35.235274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:35.301049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:35.302489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:35.387545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.398577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 652/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Thu May 19 21:51:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:51:35.404308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:35.463270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.500966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74b166-6c81-42f1-b917-8a8fa5acae7c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:35.533751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:35.534516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:35.534796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:35.535068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:35.544836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:35.546555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:51:35.564797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:35.565868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:35.566825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:51:35.566825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:51:35.566825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:51:35.566825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:35.566825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 631/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:35.571647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:35.669612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccc64-f91c-49c6-a740-75f207f425ee None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:35.702955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:35.703595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:35.703877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:35.704156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:35.704901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:35.705211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:35.706571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:35.717678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 653/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Thu May 19 21:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:35.723271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:35.822297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ffa6e-14ec-436c-b5e8-761128cc31cf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:35.836168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.856198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:35.857013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:35.857324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:35.857600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:35.864650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:35.866115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:51:35.880290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:35.881379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 21:51:35.887237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 632/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Thu May 19 21:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:35.887858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:35.937785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:35.987308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ce45-6863-4076-bfd9-193d9060fadd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:36.019975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:36.020654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:36.020947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:36.021229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:36.022020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:36.022332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:36.023757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed7a05-c57f-4241-a9c9-37f50e7fb24c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:36.036101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 654/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Thu May 19 21:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:36.042021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:36.141350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63de0-b006-4b21-bd7c-4a67dbee859e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:36.174852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:36.175492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:36.175777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:36.176054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:36.177041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] RBAC: Adding query filter params (role.id=6a9f236c49c24283afb390181f99e500, scope.project.id=df7a31e797b24706a0b85db1ee03cc75, user.id=fb31582d6fc14a7f9ec20d107c445390) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:36.177356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:36.178793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34590d2-947f-4384-800c-d77c828a3031 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:36.184504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 633/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Thu May 19 21:51:36 2022] GET /identity/v3/role_assignments?user.id=fb31582d6fc14a7f9ec20d107c445390&scope.project.id=df7a31e797b24706a0b85db1ee03cc75&role.id=6a9f236c49c24283afb390181f99e500 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:38.479419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e38d82-0c43-49e9-ae3b-ddb1a57c3c66 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:38.479948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e38d82-0c43-49e9-ae3b-ddb1a57c3c66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:38.479948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e38d82-0c43-49e9-ae3b-ddb1a57c3c66 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:38.481374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 655/1288] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:38.486994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:38.487384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:38.487558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:38.504114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:38.505337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:38.534796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:38.571060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 634/1289] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:38.593075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e38d82-0c43-49e9-ae3b-ddb1a57c3c66 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:38.692878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e38d82-0c43-49e9-ae3b-ddb1a57c3c66 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:38.703710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:38.726924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:38.727969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:38.728256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:38.728536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:38.729220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:38.730638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:38.743109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:38.744851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:38.745986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 656/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:51:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:38.751241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:38.857514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb9b08-d5e5-4e88-a2e3-ecbdfe5999ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:38.891578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:38.892443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:38.892768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:38.893053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:38.893764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:38.894071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:38.895471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0baf11-af19-4960-85bb-638a08322851 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:38.900398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 635/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Thu May 19 21:51:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:38.906891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:39.007370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f3094-1d50-4ecf-ba4d-289f225e3b96 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:39.046403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:39.047070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:39.047359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:39.047643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:39.048360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] RBAC: Adding query filter params (type=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:39.048714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:39.050086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976fcb4-4aff-48d0-a17c-d1dac39ec1af None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:39.054533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 657/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Thu May 19 21:51:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:41.245916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3673-fcaa-4db9-9380-11fe012d83ec None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:41.246476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3673-fcaa-4db9-9380-11fe012d83ec None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:41.246476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3673-fcaa-4db9-9380-11fe012d83ec None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:41.247824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 636/1293] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:41.253817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445bdb-7a8b-4680-8478-13f6aed1c443 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:41.254220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445bdb-7a8b-4680-8478-13f6aed1c443 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:41.254381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445bdb-7a8b-4680-8478-13f6aed1c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:41.271076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a9445bdb-7a8b-4680-8478-13f6aed1c443 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:41.272263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9445bdb-7a8b-4680-8478-13f6aed1c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:41.301294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:41.338666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 658/1294] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:41.362260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b3673-fcaa-4db9-9380-11fe012d83ec None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:41.393481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:41.462987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b3673-fcaa-4db9-9380-11fe012d83ec None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:41.497006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:41.497655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:41.497940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:41.498219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:41.499041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:41.500428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff44145-0ea6-4e42-9eea-60a4c9af075f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:41.513032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 637/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Thu May 19 21:51:41 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:43.748703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295957fc-6ce4-4335-a947-07acf93c2f65 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:43.749219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295957fc-6ce4-4335-a947-07acf93c2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:43.749328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295957fc-6ce4-4335-a947-07acf93c2f65 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:43.751067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 659/1296] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:43.757825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:43.758386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:43.758386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:43.775849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:43.777135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:43.848809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 638/1297] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:43.872120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295957fc-6ce4-4335-a947-07acf93c2f65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:43.971707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295957fc-6ce4-4335-a947-07acf93c2f65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:44.006047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:44.006774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:44.007055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:44.007329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:44.007977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:44.009362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:44.022440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:44.023675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:44.024811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 660/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:51:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:44.030384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:44.048866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:44.128529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc897ec5-1c98-4fea-862d-e380c38db79e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:44.161284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:44.161945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:44.162227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:44.162498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:44.163195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:44.163499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:44.164948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:44.170548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 639/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Thu May 19 21:51:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:44.176549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:44.214608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:44.283070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20835-105c-4824-bb09-61d1e3655768 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:44.318307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:44.318960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:44.319250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:44.319530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:44.320322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] RBAC: Adding query filter params (interface=public, service_id=bb19799bcf7b4ec79bd13947dc8c47b7, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:44.320654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:44.322072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0556bd9-55e3-4add-8e45-b4b9713fec70 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:44.327317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 661/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Thu May 19 21:51:44 2022] GET /identity/v3/endpoints?service_id=bb19799bcf7b4ec79bd13947dc8c47b7&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:44.333056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:44.414497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:44.433468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb1b5-ce4b-46aa-bb3c-ef4df4a57266 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:44.466581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:44.467840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:44.468156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:44.468434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:44.469143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:44.470516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f84c2-a94e-4e18-bcfe-821aee390096 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:44.477253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 640/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Thu May 19 21:51:44 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:46.664320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a4fc-46fb-4610-b440-0e5b7672413f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:46.664897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a4fc-46fb-4610-b440-0e5b7672413f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:46.664897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823a4fc-46fb-4610-b440-0e5b7672413f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:46.666283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 662/1302] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:51:46.671604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:46.672003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:46.672232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:46.689221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:46.690413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:46.755285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 641/1303] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:51:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:51:46.778066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823a4fc-46fb-4610-b440-0e5b7672413f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:46.871974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:46.879328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823a4fc-46fb-4610-b440-0e5b7672413f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:46.913059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:46.913877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:46.914166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:46.914495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:46.915094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:46.916450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:51:46.928767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:51:46.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application May 19 21:51:46.931098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 663/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Thu May 19 21:51:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:51:46.935991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:47.009349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:47.035355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3b7b-e61a-4a74-9045-46d2eff0d316 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:47.067760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:47.068412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:47.068724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:47.069002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:47.069701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:47.070002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:47.071377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043102b4-24ce-493e-a125-330f76e36c4d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:47.076937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 642/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Thu May 19 21:51:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:47.082722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:47.185304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ccecb-b5ae-4d69-b9a0-deba50afc95a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:47.220449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:47.221135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:47.221422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:47.221699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:47.222450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:47.223810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caafc7b-d482-4007-983d-49c8ab5e80bc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:47.242743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 664/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Thu May 19 21:51:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:51:57.073842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:57.238143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:51.431951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1777-c8e2-45d2-b9b0-e0dbd6e16315 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.432529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1777-c8e2-45d2-b9b0-e0dbd6e16315 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.432529 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d1777-c8e2-45d2-b9b0-e0dbd6e16315 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.438199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 643/1307] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:52:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:52:51.439511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e729-25a2-481f-a72e-cec88ad406f4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.439926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e729-25a2-481f-a72e-cec88ad406f4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.440092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e729-25a2-481f-a72e-cec88ad406f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.549975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a4e9e729-25a2-481f-a72e-cec88ad406f4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:51.551328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a4e9e729-25a2-481f-a72e-cec88ad406f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:51.689307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 665/1308] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:52:51.728097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.728384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.728769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.730900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 644/1309] 10.222.0.61 () {54 vars in 964 bytes} [Thu May 19 21:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:52:51.737502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453a954-7799-4b68-822b-84f7547d8a83 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.737841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453a954-7799-4b68-822b-84f7547d8a83 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.738073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453a954-7799-4b68-822b-84f7547d8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.851111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b453a954-7799-4b68-822b-84f7547d8a83 None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:51.852748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b453a954-7799-4b68-822b-84f7547d8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:51.876974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:51.939750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 666/1310] 10.222.0.61 () {58 vars in 1061 bytes} [Thu May 19 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:51.948854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:51.950726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:52.073890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70c1a8-8de0-4000-9d61-4efa87f8e9c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:52.158953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:52.159609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:52.159888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:52.160160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:52.161851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:52.219954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:52.280358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:52.281949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fbb9e-197a-4f8b-8ee3-0e94a5f44e16 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:52.302637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 645/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:54.824356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:54.824948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:54.825200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:54.826666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 667/1312] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:52:54.831895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042cba-4367-4d89-9852-f9dd380ee953 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:54.832320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042cba-4367-4d89-9852-f9dd380ee953 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:54.832451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042cba-4367-4d89-9852-f9dd380ee953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:54.907462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-85042cba-4367-4d89-9852-f9dd380ee953 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:54.908586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-85042cba-4367-4d89-9852-f9dd380ee953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:55.010348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 646/1313] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:55.048501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:55.050138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:55.161026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ab333-92a5-40a0-b5d5-652f8b651564 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:55.238995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:55.242010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:55.242907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:55.243422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:55.243933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:55.246338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:55.356535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:55.358048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b50848-c33f-45a8-abe9-51efb1fc9f6c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:55.377417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 668/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:57.952854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0851f-d524-41f6-b32a-e49781da14c3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:57.953418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0851f-d524-41f6-b32a-e49781da14c3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:57.953418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0851f-d524-41f6-b32a-e49781da14c3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:57.954776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 647/1315] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:52:57.960080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7552fd-c848-405d-a668-34baadf5fb98 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:57.960475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7552fd-c848-405d-a668-34baadf5fb98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:57.960678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7552fd-c848-405d-a668-34baadf5fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:58.027293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9d7552fd-c848-405d-a668-34baadf5fb98 None None] MFA Rules not processed for user `7cd78c6ca86e4f6e93ecc5bd9b90cb02`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:58.028536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9d7552fd-c848-405d-a668-34baadf5fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:58.081530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:58.126581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 669/1316] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:58.181266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:58.181682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:58.182032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:58.184045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 648/1317] 10.222.0.61 () {54 vars in 964 bytes} [Thu May 19 21:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:52:58.190146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d9c2-f3ff-4b3c-af17-a040dc3cabd7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:58.190438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d9c2-f3ff-4b3c-af17-a040dc3cabd7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:58.190717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d9c2-f3ff-4b3c-af17-a040dc3cabd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:58.268375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4cd6d9c2-f3ff-4b3c-af17-a040dc3cabd7 None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:58.269634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4cd6d9c2-f3ff-4b3c-af17-a040dc3cabd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:58.349115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 670/1318] 10.222.0.61 () {58 vars in 1061 bytes} [Thu May 19 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:52:58.358660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:58.360199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:58.370101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:58.466241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9e6a2-2cfb-462b-b49d-be3255335213 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:58.541698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:58.542841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:58.543125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:58.543402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:58.545085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:58.611857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:58.659811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:58.661290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ec90a-466b-4ef1-9ed2-7d99e527e24a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:58.679557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 649/1319] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:52:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:01.844708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:01.845244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:01.845340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:01.846649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 671/1320] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:01.851824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea8237-a41a-4613-b573-f3f4a136e1bc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:01.852500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea8237-a41a-4613-b573-f3f4a136e1bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:01.852691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea8237-a41a-4613-b573-f3f4a136e1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:01.916787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d4ea8237-a41a-4613-b573-f3f4a136e1bc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:01.918231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4ea8237-a41a-4613-b573-f3f4a136e1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:02.014668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 650/1321] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:02.053148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:02.054730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:02.107077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:02.160572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b3967-2118-49d9-be04-04140b3aa29b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:02.253729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:02.254374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:02.254659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:02.254936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:02.256573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:02.352722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:02.366628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:02.368001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45098083-d55a-4de5-99af-c3880c6ed6ee service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:02.386430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 672/1322] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:05.362552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:05.363144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:05.363144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:05.364654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 651/1323] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:05.369705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c46cb-1eef-4b9c-98d5-36106423d89e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:05.370145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c46cb-1eef-4b9c-98d5-36106423d89e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:05.370341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c46cb-1eef-4b9c-98d5-36106423d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:05.435325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7e0c46cb-1eef-4b9c-98d5-36106423d89e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:05.436523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e0c46cb-1eef-4b9c-98d5-36106423d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:05.532901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 673/1324] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:05.569206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:05.570770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:05.593294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:05.681853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a5742-6a0a-484b-a2b3-7bf598a0c4d3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:05.758295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:05.758946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:05.759227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:05.759499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:05.761760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:05.834274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:05.874120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:05.875601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb1557-a667-41f8-8060-238068dc4538 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:05.896320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 652/1325] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:08.191208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:08.191789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:08.191789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:08.193230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 674/1326] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:08.199128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da2f82-6ce0-4561-9372-01232988eb43 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:08.199546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da2f82-6ce0-4561-9372-01232988eb43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:08.199758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da2f82-6ce0-4561-9372-01232988eb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:08.264976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15da2f82-6ce0-4561-9372-01232988eb43 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:08.266198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15da2f82-6ce0-4561-9372-01232988eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:08.363466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 653/1327] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:08.399576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:08.401189 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:08.461880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:08.505260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1508da-cbbb-4b90-84b2-913fdc7e8720 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:08.578565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:08.579214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:08.579502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:08.579783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:08.581452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:08.688564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:08.691250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:08.692819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aa8cc-d25d-4c9a-a726-8a1a938b49f5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:08.711740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 675/1328] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:11.085198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:11.085722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:11.085844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:11.087126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 654/1329] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:11.092862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dbdfe-e088-45ec-8608-fb3341eb9cd5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:11.093244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dbdfe-e088-45ec-8608-fb3341eb9cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:11.093434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dbdfe-e088-45ec-8608-fb3341eb9cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:11.171019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d3dbdfe-e088-45ec-8608-fb3341eb9cd5 None None] MFA Rules not processed for user `7cd78c6ca86e4f6e93ecc5bd9b90cb02`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:11.172515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d3dbdfe-e088-45ec-8608-fb3341eb9cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:11.291514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 676/1330] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:11.329051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:11.330652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:11.361655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:11.434913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc417-d1e7-465a-b4dd-014d7a6113af None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:11.506620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:11.507268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:11.507554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:11.507835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:11.509497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:11.594730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:11.627812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:11.629307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19364b-bba0-4182-9a52-9045df9d31b1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:11.647305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 655/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:14.146657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409383be-dcce-40d8-85e5-44e866459794 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:14.147185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409383be-dcce-40d8-85e5-44e866459794 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:14.147185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409383be-dcce-40d8-85e5-44e866459794 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:14.148610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 677/1332] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:14.154100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2a6c-1bf0-4563-b4e6-917b643b2f1e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:14.154519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2a6c-1bf0-4563-b4e6-917b643b2f1e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:14.154668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2a6c-1bf0-4563-b4e6-917b643b2f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:14.221264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-538a2a6c-1bf0-4563-b4e6-917b643b2f1e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:14.222631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-538a2a6c-1bf0-4563-b4e6-917b643b2f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:14.321002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 656/1333] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:14.357299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409383be-dcce-40d8-85e5-44e866459794 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:14.358817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-409383be-dcce-40d8-85e5-44e866459794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:14.428576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:14.468544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409383be-dcce-40d8-85e5-44e866459794 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:14.547684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:14.548345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:14.548707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:14.548935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:14.550613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:14.670734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:14.672196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ddc0c-df15-4714-8505-1e0a232fd4cd service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:14.683515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:14.695812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 678/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:16.989787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:16.990345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:16.990345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:16.991758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 657/1335] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:16.997246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429d57-a042-442a-9c66-fb5839a75398 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:16.997658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429d57-a042-442a-9c66-fb5839a75398 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:16.997820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429d57-a042-442a-9c66-fb5839a75398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:17.063555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3c429d57-a042-442a-9c66-fb5839a75398 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:17.064824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c429d57-a042-442a-9c66-fb5839a75398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:17.162351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 679/1336] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:17.204288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:17.205869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:17.249382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:17.316669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a71552-3bd2-450f-a93e-7f47a71b457a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:17.389687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:17.390332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:17.390619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:17.390892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:17.392526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:17.472555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:17.498874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:17.500249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e1776-0c2b-4d54-989f-ffec145e75b0 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:17.517825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 658/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:20.110990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:20.111528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:20.111528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:20.112978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 680/1338] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:20.118174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a01b7-8f86-4064-96a5-0ab58d66728a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:20.118589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a01b7-8f86-4064-96a5-0ab58d66728a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:20.118742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a01b7-8f86-4064-96a5-0ab58d66728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:20.183838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-417a01b7-8f86-4064-96a5-0ab58d66728a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:20.185085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-417a01b7-8f86-4064-96a5-0ab58d66728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:20.286201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 659/1339] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:53:20.321392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:20.322952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:20.399422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:20.427680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1fe38-41c6-4210-b023-8c920d446213 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:20.501532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:20.502462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:20.502749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:20.503053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:20.504735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:20.615132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:20.616574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b50d-18ea-4cfe-b9d6-35f16d35e72b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:20.634412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 681/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:24.927806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:24.928399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:24.928399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:24.929753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 660/1341] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:24.935567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993fa1-4732-46b4-8b99-7e2848d151d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:24.935983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993fa1-4732-46b4-8b99-7e2848d151d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:24.936129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993fa1-4732-46b4-8b99-7e2848d151d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:24.942138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:25.007718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d8993fa1-4732-46b4-8b99-7e2848d151d3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:25.008847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8993fa1-4732-46b4-8b99-7e2848d151d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:25.125834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 682/1342] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:25.164286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:25.166468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:25.219203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:25.269568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92a8e-9684-45e4-b2c3-ab87ce89d51f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:25.275352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:25.276067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:25.276376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:25.276705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:25.287070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:25.289157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903daf-bda6-4113-aaea-43efff4b2f7e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:25.302151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 661/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:28.269607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4810c6-be57-477e-bc47-acae28f23105 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:28.270132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4810c6-be57-477e-bc47-acae28f23105 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:28.270132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4810c6-be57-477e-bc47-acae28f23105 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:28.271508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 683/1344] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:28.277541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dc40-d4c1-4007-992b-945ee655bfaa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:28.277938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dc40-d4c1-4007-992b-945ee655bfaa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:28.278097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dc40-d4c1-4007-992b-945ee655bfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:28.305776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-33e0dc40-d4c1-4007-992b-945ee655bfaa None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:28.306969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-33e0dc40-d4c1-4007-992b-945ee655bfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:28.382858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 662/1345] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:28.419559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4810c6-be57-477e-bc47-acae28f23105 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:28.430200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4810c6-be57-477e-bc47-acae28f23105 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:28.434320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:28.434958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:28.435243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:28.435522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:28.444766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:28.446203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ce3a8-f713-4e36-b194-35ffaeb1dc5a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:28.455750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 684/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:32.835695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:32.836251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:32.836348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:32.837733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 663/1347] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:32.843058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f0f4-5db4-41b5-be1d-1e60e588965e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:32.843466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f0f4-5db4-41b5-be1d-1e60e588965e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:32.843632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f0f4-5db4-41b5-be1d-1e60e588965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:32.862985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0900f0f4-5db4-41b5-be1d-1e60e588965e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:32.864223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0900f0f4-5db4-41b5-be1d-1e60e588965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:32.872697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 685/1348] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:32.909861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:32.911557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:32.924961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb98e6-fb2f-41fc-a147-182598191840 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:32.928517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:32.929251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:32.929606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:32.929891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:32.939972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:32.941461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9972eac-0426-437e-b60f-c878ea5b8805 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:32.951437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 664/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:38.406751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c5-d509-4906-8303-e78739a2a4d3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:38.407590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c5-d509-4906-8303-e78739a2a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:38.408050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad027c5-d509-4906-8303-e78739a2a4d3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:38.409975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 686/1350] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 21:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:38.414887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5adc79-86c5-42a2-8061-ed4e26447a27 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:38.415624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5adc79-86c5-42a2-8061-ed4e26447a27 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:38.416054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5adc79-86c5-42a2-8061-ed4e26447a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:38.488990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5c5adc79-86c5-42a2-8061-ed4e26447a27 None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:38.490319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c5adc79-86c5-42a2-8061-ed4e26447a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:38.521480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:38.576146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 665/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 21:53:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:42.857885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=72 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:43.077834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1021-34b5-4270-81bc-b9cff776ebd9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:43.078117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1021-34b5-4270-81bc-b9cff776ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:43.078592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1021-34b5-4270-81bc-b9cff776ebd9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:43.079966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 687/1352] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 21:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:43.085221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480f06c-270d-4fa5-909e-054a98ff1a3e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:43.085679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480f06c-270d-4fa5-909e-054a98ff1a3e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:43.086391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480f06c-270d-4fa5-909e-054a98ff1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:43.108154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6480f06c-270d-4fa5-909e-054a98ff1a3e None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:43.109395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6480f06c-270d-4fa5-909e-054a98ff1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:43.118423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 666/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 21:53:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:44.653315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b92ad-d447-45d8-b21c-6d0443a579d9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:44.653846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b92ad-d447-45d8-b21c-6d0443a579d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:44.653846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b92ad-d447-45d8-b21c-6d0443a579d9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:44.655228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 688/1354] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 21:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:44.660321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688323-f636-474f-a875-2ece242e3a5d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:44.660796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688323-f636-474f-a875-2ece242e3a5d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:44.660949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688323-f636-474f-a875-2ece242e3a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:44.680359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-32688323-f636-474f-a875-2ece242e3a5d None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:44.681571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-32688323-f636-474f-a875-2ece242e3a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:44.690702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 667/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 21:53:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:48.519059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4e4b-5547-4148-a935-22c0c851cda8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:48.519625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4e4b-5547-4148-a935-22c0c851cda8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:48.519625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4e4b-5547-4148-a935-22c0c851cda8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:48.520993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 689/1356] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:48.526525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8832f4-6884-438f-99b8-469209ded748 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:48.526952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8832f4-6884-438f-99b8-469209ded748 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:48.527147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8832f4-6884-438f-99b8-469209ded748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:48.546083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6b8832f4-6884-438f-99b8-469209ded748 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:48.547214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6b8832f4-6884-438f-99b8-469209ded748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:48.555406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 668/1357] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:48.611156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef5d8-8776-4aec-a7b6-d334307facd5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:48.611777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef5d8-8776-4aec-a7b6-d334307facd5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:48.612220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef5d8-8776-4aec-a7b6-d334307facd5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:48.613983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 690/1358] 10.222.0.61 () {54 vars in 961 bytes} [Thu May 19 21:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:48.620646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053140c-de21-49f0-af64-632d8262211c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:48.620928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053140c-de21-49f0-af64-632d8262211c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:48.621209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053140c-de21-49f0-af64-632d8262211c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:48.678794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f053140c-de21-49f0-af64-632d8262211c None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:48.679978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f053140c-de21-49f0-af64-632d8262211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:48.770596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 669/1359] 10.222.0.61 () {58 vars in 1058 bytes} [Thu May 19 21:53:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 19 21:53:48.791857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef5d8-8776-4aec-a7b6-d334307facd5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:48.807176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef5d8-8776-4aec-a7b6-d334307facd5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:48.813018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:48.813787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:48.814167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:48.814541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:48.823762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:48.826463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412df0b4-c682-48f4-ae10-242934c91fc9 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:48.839995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 691/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:53:51.323430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647393-622f-49fb-9fdf-fa112bc89654 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:51.323965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647393-622f-49fb-9fdf-fa112bc89654 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:51.323965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647393-622f-49fb-9fdf-fa112bc89654 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:51.325530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 670/1361] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:51.330749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749da13-ada4-41a1-aa0d-0f10796c6658 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:51.331166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749da13-ada4-41a1-aa0d-0f10796c6658 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:51.331311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749da13-ada4-41a1-aa0d-0f10796c6658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:51.350373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a749da13-ada4-41a1-aa0d-0f10796c6658 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:51.351578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a749da13-ada4-41a1-aa0d-0f10796c6658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:51.360720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 692/1362] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:51.405530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34647393-622f-49fb-9fdf-fa112bc89654 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:51.421518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34647393-622f-49fb-9fdf-fa112bc89654 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:51.426967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:51.427900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:51.428343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:51.428820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:51.438216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:51.442313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b263f6-1c1b-4f65-8a33-32de0e8f6cf2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:51.455330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 671/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:54.053788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bd03-069a-4a46-8f89-c75777dccdae None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:54.054302 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bd03-069a-4a46-8f89-c75777dccdae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:54.054414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bd03-069a-4a46-8f89-c75777dccdae None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:54.055719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 693/1364] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:54.061115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1670fb-5b3a-4858-855f-cb1a6d36581e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:54.061961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1670fb-5b3a-4858-855f-cb1a6d36581e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:54.062244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1670fb-5b3a-4858-855f-cb1a6d36581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:54.081748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-db1670fb-5b3a-4858-855f-cb1a6d36581e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:54.082396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db1670fb-5b3a-4858-855f-cb1a6d36581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:54.090951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 672/1365] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:54.137237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ad95-0076-4f2d-9b1e-c3f17e3e882a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:54.137530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ad95-0076-4f2d-9b1e-c3f17e3e882a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:54.137815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077ad95-0076-4f2d-9b1e-c3f17e3e882a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:54.139237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 694/1366] 10.222.0.61 () {54 vars in 961 bytes} [Thu May 19 21:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:54.144481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248561-b57e-4cdb-b271-354830c68591 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:54.145173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248561-b57e-4cdb-b271-354830c68591 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:54.145457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248561-b57e-4cdb-b271-354830c68591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:54.167471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7248561-b57e-4cdb-b271-354830c68591 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:54.168692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7248561-b57e-4cdb-b271-354830c68591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:54.187688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 673/1367] 10.222.0.61 () {58 vars in 1058 bytes} [Thu May 19 21:53:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:54.195927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077ad95-0076-4f2d-9b1e-c3f17e3e882a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:54.216362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077ad95-0076-4f2d-9b1e-c3f17e3e882a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:54.221932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:54.222525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:54.222817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:54.223096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:54.231731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:54.233356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31783ae3-5538-4d95-8815-036bba38caeb service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:54.242744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 695/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:56.549331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748dd3-6849-4507-928e-84147132a7ab None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:56.549893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748dd3-6849-4507-928e-84147132a7ab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:56.549893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748dd3-6849-4507-928e-84147132a7ab None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:56.551293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 674/1369] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:56.557350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3e05-8ca4-4574-9f11-0aee36412a9f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:56.557795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3e05-8ca4-4574-9f11-0aee36412a9f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:56.557927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3e05-8ca4-4574-9f11-0aee36412a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:56.577098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d94b3e05-8ca4-4574-9f11-0aee36412a9f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:56.578368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d94b3e05-8ca4-4574-9f11-0aee36412a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:56.587544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 696/1370] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:56.631768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53748dd3-6849-4507-928e-84147132a7ab None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:56.647151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53748dd3-6849-4507-928e-84147132a7ab None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:56.652723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:56.653374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:56.653659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:56.653941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:56.662736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:56.664352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7505f8c-71bc-4a35-b4b8-2d5b4b2d3f4e service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:56.674557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 675/1371] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:56.941047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbd11e-786e-4509-8014-406a4b74fc5d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:56.941263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbd11e-786e-4509-8014-406a4b74fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:56.941733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbd11e-786e-4509-8014-406a4b74fc5d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:56.944147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 697/1372] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:56.949513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267da3e-7a5b-42a5-8b7e-fbafe03b5da7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:56.950017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267da3e-7a5b-42a5-8b7e-fbafe03b5da7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:56.950460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267da3e-7a5b-42a5-8b7e-fbafe03b5da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:56.970404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5267da3e-7a5b-42a5-8b7e-fbafe03b5da7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:56.971646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5267da3e-7a5b-42a5-8b7e-fbafe03b5da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:56.980158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 676/1373] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:57.292502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0800ef-6ef0-4e95-864d-229fee0e1284 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:57.292502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0800ef-6ef0-4e95-864d-229fee0e1284 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:57.292502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0800ef-6ef0-4e95-864d-229fee0e1284 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:57.293814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 698/1374] 10.222.0.61 () {54 vars in 963 bytes} [Thu May 19 21:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:57.298797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b89a-32f8-4a72-b54a-6f02e60c3cc8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:57.299095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b89a-32f8-4a72-b54a-6f02e60c3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:57.299381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b89a-32f8-4a72-b54a-6f02e60c3cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:57.307439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:57.352936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0842b89a-32f8-4a72-b54a-6f02e60c3cc8 None None] MFA Rules not processed for user `41cfa205823a4c29abea3001bf086711`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:57.354929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0842b89a-32f8-4a72-b54a-6f02e60c3cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:57.417924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 677/1375] 10.222.0.61 () {58 vars in 1060 bytes} [Thu May 19 21:53:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:57.425480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0800ef-6ef0-4e95-864d-229fee0e1284 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:57.436490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0800ef-6ef0-4e95-864d-229fee0e1284 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:57.440047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:57.440802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:57.441096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:57.441381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:57.451794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:57.453311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b06c9-89a5-4380-bcaf-6d82951d502d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:57.463892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 699/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:58.993553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ab37-be1f-4805-9b4f-254c41b796f6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:58.994672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ab37-be1f-4805-9b4f-254c41b796f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:58.994907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ab37-be1f-4805-9b4f-254c41b796f6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:58.996666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 678/1377] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:53:59.002690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791f05-2054-4c03-ab11-0a7ce6aca8cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:59.003266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791f05-2054-4c03-ab11-0a7ce6aca8cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:59.003266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d791f05-2054-4c03-ab11-0a7ce6aca8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:59.027839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4d791f05-2054-4c03-ab11-0a7ce6aca8cc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:53:59.027839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d791f05-2054-4c03-ab11-0a7ce6aca8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:59.035368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 700/1378] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:59.080675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3ab37-be1f-4805-9b4f-254c41b796f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:59.096652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3ab37-be1f-4805-9b4f-254c41b796f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:59.102680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:59.103932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:59.104790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:59.104928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:59.114385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:59.116043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc44043-2410-4eaf-9e1e-32ab8fedb2b6 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:59.128425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 679/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:01.409014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d1f8-d3d5-4368-8fb9-0527721c623f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:01.409905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d1f8-d3d5-4368-8fb9-0527721c623f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:01.410358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d1f8-d3d5-4368-8fb9-0527721c623f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:01.411923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 701/1380] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:01.418303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc5447-9e86-4880-9788-8f1c949bee79 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:01.418741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc5447-9e86-4880-9788-8f1c949bee79 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:01.418867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc5447-9e86-4880-9788-8f1c949bee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:01.437283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d6fc5447-9e86-4880-9788-8f1c949bee79 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:01.438491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d6fc5447-9e86-4880-9788-8f1c949bee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:01.446535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 680/1381] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:01.491490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d1f8-d3d5-4368-8fb9-0527721c623f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:01.507213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d1f8-d3d5-4368-8fb9-0527721c623f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:01.512943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:01.513696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:01.514105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:01.514486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:01.523196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:01.525440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089c28b-21e5-4915-a01f-9ba7b5df41c7 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:01.535769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 702/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:03.829271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3890e-ca64-45d3-afc3-2d488d8f1668 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:03.830174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3890e-ca64-45d3-afc3-2d488d8f1668 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:03.830612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3890e-ca64-45d3-afc3-2d488d8f1668 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:03.837861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb2b3-55d4-46f0-a433-6c0099e8a3e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:03.838843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb2b3-55d4-46f0-a433-6c0099e8a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:03.839274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb2b3-55d4-46f0-a433-6c0099e8a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:03.841083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 681/1383] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:03.858572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-643eb2b3-55d4-46f0-a433-6c0099e8a3e9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:03.859799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-643eb2b3-55d4-46f0-a433-6c0099e8a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:03.879135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 703/1384] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:03.910356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3890e-ca64-45d3-afc3-2d488d8f1668 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:03.925860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3890e-ca64-45d3-afc3-2d488d8f1668 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:03.932186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:03.932939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:03.933230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:03.933514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:03.943197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:03.944965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378c201-b5d8-4780-98b1-936e78cab9b6 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:03.956305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 682/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:06.232300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ba0-12f9-4a86-b1e2-0fb96009ca07 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:06.232976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ba0-12f9-4a86-b1e2-0fb96009ca07 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:06.233797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ba0-12f9-4a86-b1e2-0fb96009ca07 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:06.235909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 704/1386] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:06.243741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add372fe-0586-4346-b77c-c92d1b5ab858 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:06.244289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add372fe-0586-4346-b77c-c92d1b5ab858 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:06.244716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add372fe-0586-4346-b77c-c92d1b5ab858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:06.266087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-add372fe-0586-4346-b77c-c92d1b5ab858 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:06.267276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-add372fe-0586-4346-b77c-c92d1b5ab858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:06.275857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 683/1387] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:06.330313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175ba0-12f9-4a86-b1e2-0fb96009ca07 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:06.345388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175ba0-12f9-4a86-b1e2-0fb96009ca07 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:06.350777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:06.351414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:06.351701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:06.351982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:06.360823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:06.362413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b22ff-430f-4d23-ab01-3f587d20ab67 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:06.372137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 705/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:08.637310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3140-b201-4263-beb6-9c423bb0e56d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:08.638085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3140-b201-4263-beb6-9c423bb0e56d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:08.638085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3140-b201-4263-beb6-9c423bb0e56d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:08.639253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 684/1389] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:54:08.644723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e19e-dbb3-4277-89a6-daaff00d9ac5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:08.645484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e19e-dbb3-4277-89a6-daaff00d9ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:08.645804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e19e-dbb3-4277-89a6-daaff00d9ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:08.664393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5662e19e-dbb3-4277-89a6-daaff00d9ac5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:08.665683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5662e19e-dbb3-4277-89a6-daaff00d9ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:08.673716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 706/1390] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:08.719187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3140-b201-4263-beb6-9c423bb0e56d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:08.734521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3140-b201-4263-beb6-9c423bb0e56d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:08.740127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:08.740762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:08.741110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:08.741331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:08.750178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:08.751799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c64fa6-4178-45d1-8d5c-bfe30c14c21d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:08.767081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 685/1391] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:11.079241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda00c-eaf5-4115-b0e1-57b77947716e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:11.079885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda00c-eaf5-4115-b0e1-57b77947716e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:11.079885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfda00c-eaf5-4115-b0e1-57b77947716e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:11.081202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 707/1392] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:11.086323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76480ca6-9f71-4e25-bf16-b0fb63d47cf8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:11.086762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76480ca6-9f71-4e25-bf16-b0fb63d47cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:11.086910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76480ca6-9f71-4e25-bf16-b0fb63d47cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:11.105656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-76480ca6-9f71-4e25-bf16-b0fb63d47cf8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:11.106864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-76480ca6-9f71-4e25-bf16-b0fb63d47cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:11.114714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 686/1393] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:11.180110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfda00c-eaf5-4115-b0e1-57b77947716e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:11.198274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfda00c-eaf5-4115-b0e1-57b77947716e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:11.204519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:11.205206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:11.205494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:11.205768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:11.214330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:11.215918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e349559-7940-4cee-889d-7abc3b3821d2 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:11.225733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 708/1394] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:13.402740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff0a6a-8805-47c5-8ea9-3e021ed88278 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.403311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff0a6a-8805-47c5-8ea9-3e021ed88278 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.403311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff0a6a-8805-47c5-8ea9-3e021ed88278 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.404739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 687/1395] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:13.410723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0524f-6e3b-4a68-861a-a5d8904dda98 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.411179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0524f-6e3b-4a68-861a-a5d8904dda98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.411287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0524f-6e3b-4a68-861a-a5d8904dda98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.417377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=52 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:54:13.430218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-38b0524f-6e3b-4a68-861a-a5d8904dda98 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:13.432183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38b0524f-6e3b-4a68-861a-a5d8904dda98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:13.440035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 709/1396] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:13.497707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41731-743a-470e-b0ef-a9ee55885299 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.498001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41731-743a-470e-b0ef-a9ee55885299 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.498286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41731-743a-470e-b0ef-a9ee55885299 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.499881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 688/1397] 10.222.0.61 () {54 vars in 962 bytes} [Thu May 19 21:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:13.501348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e456-efa1-4d42-8a62-38d658519755 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.501704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e456-efa1-4d42-8a62-38d658519755 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.502050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15e456-efa1-4d42-8a62-38d658519755 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.503502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 710/1398] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:54:13.504830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b524b61-55fa-4b1c-aa21-1290c0c8767b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.506238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b524b61-55fa-4b1c-aa21-1290c0c8767b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.506521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b524b61-55fa-4b1c-aa21-1290c0c8767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.508377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.508743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.509605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.527197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:13.528472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:13.536779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 711/1399] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:13.558141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1b524b61-55fa-4b1c-aa21-1290c0c8767b None None] MFA Rules not processed for user `de01947b95e746f8a5d6f711d982f97e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:13.559354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b524b61-55fa-4b1c-aa21-1290c0c8767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:13.580375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:13.594004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b427b6-f1d2-4b5e-bde6-3cb01fb3c196 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:13.599417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:13.600065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.600350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.600661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.609800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:13.611381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:13.617148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:54:13.621850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 712/1400] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:54:13.626044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 689/1401] 10.222.0.61 () {58 vars in 1059 bytes} [Thu May 19 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:13.633799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:13.643997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a75a5-d78b-4e59-98ad-9c5196f7c6c0 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:13.647618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:13.648251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:13.648676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:13.648960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:13.659005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:13.660483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f23bbc-314f-4c53-a013-2162de93881b service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:13.670330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 713/1402] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:54:14.231531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e283fb-c7c4-48ba-a74c-cecc52901b9f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:14.231953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e283fb-c7c4-48ba-a74c-cecc52901b9f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:14.232388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e283fb-c7c4-48ba-a74c-cecc52901b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:14.286960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b3e283fb-c7c4-48ba-a74c-cecc52901b9f None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:14.288179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b3e283fb-c7c4-48ba-a74c-cecc52901b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:14.357987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 690/1403] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 21:54:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:14.401235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416917-7bb9-4e1f-bb6f-0ffcf855a4e9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:14.401699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416917-7bb9-4e1f-bb6f-0ffcf855a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:14.402113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416917-7bb9-4e1f-bb6f-0ffcf855a4e9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:14.403575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 714/1404] 10.222.0.61 () {54 vars in 942 bytes} [Thu May 19 21:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:14.410396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a0a7-3211-4ac9-8ab3-ef8ce134dc5a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:14.410877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a0a7-3211-4ac9-8ab3-ef8ce134dc5a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:14.411115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a0a7-3211-4ac9-8ab3-ef8ce134dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:14.460348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-be46a0a7-3211-4ac9-8ab3-ef8ce134dc5a None None] MFA Rules not processed for user `1d1c9cf9b02a41f39f46e878747cea3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:14.461601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-be46a0a7-3211-4ac9-8ab3-ef8ce134dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:14.533885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 691/1405] 10.222.0.61 () {58 vars in 1039 bytes} [Thu May 19 21:54:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:14.543523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416917-7bb9-4e1f-bb6f-0ffcf855a4e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:14.557680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416917-7bb9-4e1f-bb6f-0ffcf855a4e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:14.562693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:14.563322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:14.563613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:14.563892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:14.572231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:14.573889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15c650-9c3b-421f-bec4-be1678f8ecf5 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:14.582095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 715/1406] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 21:54:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:54:15.901863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cf75b-4caa-41cb-bde8-4a6dc73c9868 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:15.902387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cf75b-4caa-41cb-bde8-4a6dc73c9868 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:15.902387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cf75b-4caa-41cb-bde8-4a6dc73c9868 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:15.913295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 692/1407] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:15.915489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee9123-48f3-40a1-b320-98c74b03ce4d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:15.916733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee9123-48f3-40a1-b320-98c74b03ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:15.917059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee9123-48f3-40a1-b320-98c74b03ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:15.952556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4dee9123-48f3-40a1-b320-98c74b03ce4d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:15.955583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4dee9123-48f3-40a1-b320-98c74b03ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:15.971224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 716/1408] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:16.066093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cf75b-4caa-41cb-bde8-4a6dc73c9868 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:16.083166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cf75b-4caa-41cb-bde8-4a6dc73c9868 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:16.089116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:16.089985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:16.090575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:16.090857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:16.099966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:16.101742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7266f-d099-4d0f-806d-b7ae4518d773 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:16.113221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 693/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:25.944115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=26 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:54:26.110879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:54:37.830020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd3ce-8d90-4abc-b075-2c7ddafe3f8f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:37.830662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd3ce-8d90-4abc-b075-2c7ddafe3f8f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:37.830662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd3ce-8d90-4abc-b075-2c7ddafe3f8f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:37.831976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 717/1410] 10.222.0.61 () {54 vars in 918 bytes} [Thu May 19 21:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:37.837776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f642-c909-4fb0-a3d7-d7eaaa3fad70 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:37.838239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f642-c909-4fb0-a3d7-d7eaaa3fad70 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:37.838343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f642-c909-4fb0-a3d7-d7eaaa3fad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:37.856660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7067f642-c909-4fb0-a3d7-d7eaaa3fad70 None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:37.857869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7067f642-c909-4fb0-a3d7-d7eaaa3fad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:37.866548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 694/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Thu May 19 21:54:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:37.880266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630480e7-9689-45a2-8197-acada7cf6bff None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:37.880559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630480e7-9689-45a2-8197-acada7cf6bff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:37.880867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630480e7-9689-45a2-8197-acada7cf6bff None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:37.882223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 718/1412] 10.222.0.61 () {54 vars in 918 bytes} [Thu May 19 21:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:37.887185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a4936-ab48-4873-978d-e685b62f4622 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:37.887473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a4936-ab48-4873-978d-e685b62f4622 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:37.887753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a4936-ab48-4873-978d-e685b62f4622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:37.905245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c48a4936-ab48-4873-978d-e685b62f4622 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:37.906409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c48a4936-ab48-4873-978d-e685b62f4622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:37.915765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 695/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Thu May 19 21:54:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:40.675019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c762e48-0457-41fc-b8e4-190b0cc5ef5a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:40.675606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c762e48-0457-41fc-b8e4-190b0cc5ef5a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:40.675606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c762e48-0457-41fc-b8e4-190b0cc5ef5a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:40.676998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:40.682638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322e4e-7591-464a-adc8-a445311fcbcd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:40.683035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322e4e-7591-464a-adc8-a445311fcbcd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:40.683205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322e4e-7591-464a-adc8-a445311fcbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:40.701627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b1322e4e-7591-464a-adc8-a445311fcbcd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:40.702812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b1322e4e-7591-464a-adc8-a445311fcbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:40.710424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:40.759876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c762e48-0457-41fc-b8e4-190b0cc5ef5a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:40.776143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c762e48-0457-41fc-b8e4-190b0cc5ef5a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:40.781506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:40.782146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:40.782446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:40.782711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:40.791390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:40.793008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ab655-170e-4dab-ad1a-933b621266f5 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:40.802211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 720/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:41.809601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10760b8-7af6-4936-bb6c-b934ea04eaad None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:41.810174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10760b8-7af6-4936-bb6c-b934ea04eaad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:41.810174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10760b8-7af6-4936-bb6c-b934ea04eaad None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:41.811541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:41.816304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9cb2-dc6e-41f3-9d26-e2b890246f42 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:41.816739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9cb2-dc6e-41f3-9d26-e2b890246f42 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:41.816889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9cb2-dc6e-41f3-9d26-e2b890246f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:41.834996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8c2a9cb2-dc6e-41f3-9d26-e2b890246f42 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:41.836193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8c2a9cb2-dc6e-41f3-9d26-e2b890246f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:41.843863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:41.995545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10760b8-7af6-4936-bb6c-b934ea04eaad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:42.010663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10760b8-7af6-4936-bb6c-b934ea04eaad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:42.016072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:42.016741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:42.017031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:42.017315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:42.026344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:42.027953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f98872-74af-4197-bcfe-5aacfb174f25 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:42.037551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 698/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:42.997564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582699-09be-4856-a098-d352088a8400 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:42.998109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582699-09be-4856-a098-d352088a8400 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:42.998208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582699-09be-4856-a098-d352088a8400 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:42.999546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:43.004427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1ba-a794-4fc6-91b7-d25c464afc18 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:43.004850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1ba-a794-4fc6-91b7-d25c464afc18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:43.005027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1ba-a794-4fc6-91b7-d25c464afc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:43.026023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63d4d1ba-a794-4fc6-91b7-d25c464afc18 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:43.027232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63d4d1ba-a794-4fc6-91b7-d25c464afc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:43.035323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:43.185298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51582699-09be-4856-a098-d352088a8400 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:43.199941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51582699-09be-4856-a098-d352088a8400 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:43.205393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:43.206024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:43.206305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:43.206579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:43.215383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:43.216991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374257a9-315b-4ac3-bf6a-0266e9358dc1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:43.226079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 723/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:44.155797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550507f-baa4-4a17-ab10-e49fc5ba5d5b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:44.156696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550507f-baa4-4a17-ab10-e49fc5ba5d5b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:44.156987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550507f-baa4-4a17-ab10-e49fc5ba5d5b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:44.158406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:44.163304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db620-bf0c-4e33-80c3-80cabc514093 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:44.163682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db620-bf0c-4e33-80c3-80cabc514093 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:44.163871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db620-bf0c-4e33-80c3-80cabc514093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:44.182826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-df4db620-bf0c-4e33-80c3-80cabc514093 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:44.184047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df4db620-bf0c-4e33-80c3-80cabc514093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:44.192179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:44.351375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550507f-baa4-4a17-ab10-e49fc5ba5d5b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:44.366760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550507f-baa4-4a17-ab10-e49fc5ba5d5b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:44.374347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:44.374980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:44.375267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:44.375548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:44.383819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:44.385461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427ef77-5e41-4e2a-bd4e-f08b19a15093 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:44.395452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 701/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:45.339699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3edf-7c49-49f8-aea4-62c508d69651 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:45.340230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3edf-7c49-49f8-aea4-62c508d69651 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:45.340326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3edf-7c49-49f8-aea4-62c508d69651 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:45.341745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:45.346486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa059-6afb-4379-8ab1-0a1954a1156a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:45.346901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa059-6afb-4379-8ab1-0a1954a1156a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:45.347035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa059-6afb-4379-8ab1-0a1954a1156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:45.365667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-198fa059-6afb-4379-8ab1-0a1954a1156a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:45.366877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-198fa059-6afb-4379-8ab1-0a1954a1156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:45.375059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:45.540967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3edf-7c49-49f8-aea4-62c508d69651 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:45.554798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3edf-7c49-49f8-aea4-62c508d69651 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:45.560882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:45.561506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:45.561789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:45.562064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:45.570098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:45.572392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd67c9-507b-42b7-b145-03211021863e service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:45.581575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 726/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:46.466338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8126b8-f3f3-474e-8800-56f3428e19a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:46.466877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8126b8-f3f3-474e-8800-56f3428e19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:46.466877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8126b8-f3f3-474e-8800-56f3428e19a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:46.468569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:46.473432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab7a0-447d-42b9-9171-9460dafc4613 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:46.473859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab7a0-447d-42b9-9171-9460dafc4613 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:46.474006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab7a0-447d-42b9-9171-9460dafc4613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:46.493498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5beab7a0-447d-42b9-9171-9460dafc4613 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:46.494709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5beab7a0-447d-42b9-9171-9460dafc4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:46.502505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:46.652884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8126b8-f3f3-474e-8800-56f3428e19a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:46.667827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8126b8-f3f3-474e-8800-56f3428e19a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:46.673458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:46.674085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:46.674368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:46.674644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:46.683013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:46.684592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71944f-8575-4719-93c6-19efbcfd8fb0 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:46.693820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 704/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:47.575435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf6db-3961-4577-9571-b4c377f7f05c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:47.576004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf6db-3961-4577-9571-b4c377f7f05c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:47.576004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf6db-3961-4577-9571-b4c377f7f05c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:47.577427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:47.582343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437209a5-dfa9-4367-8a05-14d9a8fab6fc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:47.582749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437209a5-dfa9-4367-8a05-14d9a8fab6fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:47.582905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437209a5-dfa9-4367-8a05-14d9a8fab6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:47.600999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-437209a5-dfa9-4367-8a05-14d9a8fab6fc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:47.602489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-437209a5-dfa9-4367-8a05-14d9a8fab6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:47.610075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:47.759388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbf6db-3961-4577-9571-b4c377f7f05c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:47.774422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbf6db-3961-4577-9571-b4c377f7f05c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:47.779764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:47.780396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:47.780714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:47.780995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:47.788926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:47.790502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c49fd-66f0-4d7c-90f5-c9b1d88a6743 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:47.799657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 729/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:48.678681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2114a-e5dc-435d-aa42-e115d5173bfe None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:48.679192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2114a-e5dc-435d-aa42-e115d5173bfe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:48.679192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2114a-e5dc-435d-aa42-e115d5173bfe None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:48.681690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:48.686531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f028-c0b1-445c-b084-dcf3e27d2754 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:48.686986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f028-c0b1-445c-b084-dcf3e27d2754 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:48.687108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f028-c0b1-445c-b084-dcf3e27d2754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:48.705483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c1d5f028-c0b1-445c-b084-dcf3e27d2754 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:48.706682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c1d5f028-c0b1-445c-b084-dcf3e27d2754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:48.714951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:48.866776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2114a-e5dc-435d-aa42-e115d5173bfe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:48.881305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2114a-e5dc-435d-aa42-e115d5173bfe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:48.886830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:48.887475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:48.887762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:48.888040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:48.896593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:48.898285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7d270-b6c5-4989-8a16-d90e21eda97f service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:48.908370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 707/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:49.783625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51244-5826-484e-a254-e631f5442281 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:49.784186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51244-5826-484e-a254-e631f5442281 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:49.784186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51244-5826-484e-a254-e631f5442281 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:49.785641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:49.790639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9e688-7ae5-4fff-baf1-790759bbadc6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:49.791096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9e688-7ae5-4fff-baf1-790759bbadc6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:49.791215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9e688-7ae5-4fff-baf1-790759bbadc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:49.808906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dec9e688-7ae5-4fff-baf1-790759bbadc6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:49.810089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dec9e688-7ae5-4fff-baf1-790759bbadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:49.817734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:49.966934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51244-5826-484e-a254-e631f5442281 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:49.981631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51244-5826-484e-a254-e631f5442281 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:49.987160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:49.987793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:49.988083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:49.988430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:49.996559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:49.998167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181ff81-4a78-471f-ad96-39b815c9d1b1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:50.007453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 732/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:52.107152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1cfa1-dd58-487a-807f-3639a2af0b00 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:52.107669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1cfa1-dd58-487a-807f-3639a2af0b00 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:52.107669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1cfa1-dd58-487a-807f-3639a2af0b00 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:52.109040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:52.114366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b49dd-17b1-43fe-b506-0a5c7ad7c003 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:52.114790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b49dd-17b1-43fe-b506-0a5c7ad7c003 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:52.114938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b49dd-17b1-43fe-b506-0a5c7ad7c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:52.133040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3a5b49dd-17b1-43fe-b506-0a5c7ad7c003 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:52.134245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a5b49dd-17b1-43fe-b506-0a5c7ad7c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:52.141909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:52.180489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1cfa1-dd58-487a-807f-3639a2af0b00 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:52.191145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1cfa1-dd58-487a-807f-3639a2af0b00 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:52.194632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:52.195273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:52.195658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:52.195947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:52.205174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:52.206626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb2b61-41c0-48ef-85d2-7f6109fd1cb1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:52.216110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 710/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:54.614474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c13f72-8451-47ca-9f2d-7e4ae2ba58ef None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:54.615114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c13f72-8451-47ca-9f2d-7e4ae2ba58ef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:54.615114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c13f72-8451-47ca-9f2d-7e4ae2ba58ef None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:54.616437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:54.622309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468dc5-752c-4478-b6db-a9e38ac9d8e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:54.622785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468dc5-752c-4478-b6db-a9e38ac9d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:54.622896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d468dc5-752c-4478-b6db-a9e38ac9d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:54.641076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3d468dc5-752c-4478-b6db-a9e38ac9d8e8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:54.642269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3d468dc5-752c-4478-b6db-a9e38ac9d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:54.650401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:54:54.695322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c13f72-8451-47ca-9f2d-7e4ae2ba58ef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:54.703942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c13f72-8451-47ca-9f2d-7e4ae2ba58ef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:54.707457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:54.708089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:54.708377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:54.708694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:54.717710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:54.719082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77516-6cf5-4c77-b662-5e8ab3dba324 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:54.728908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 735/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:57.270662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893237-9bf5-423b-af60-25e894a3d619 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:57.271198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893237-9bf5-423b-af60-25e894a3d619 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:57.271198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893237-9bf5-423b-af60-25e894a3d619 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:57.272584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:57.278535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410943-5737-45f5-96bc-17f76b468fa5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:57.278944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410943-5737-45f5-96bc-17f76b468fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:57.279126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410943-5737-45f5-96bc-17f76b468fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:57.299543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f5410943-5737-45f5-96bc-17f76b468fa5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:57.300782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5410943-5737-45f5-96bc-17f76b468fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:57.309167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:57.346453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67893237-9bf5-423b-af60-25e894a3d619 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:57.358298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67893237-9bf5-423b-af60-25e894a3d619 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:57.361784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:57.362446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:57.362731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:57.363009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:57.373082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:57.374523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dbae4-e756-4820-99ef-84fd02c84729 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:57.384782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 713/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:54:59.826050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0a74-9584-4888-8d62-1ec364020162 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:59.826619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0a74-9584-4888-8d62-1ec364020162 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:59.826619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f0a74-9584-4888-8d62-1ec364020162 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:59.828000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:54:59.833372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5bb-1672-4bc1-a87a-b6eb88eb392c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:59.833764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5bb-1672-4bc1-a87a-b6eb88eb392c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:59.833957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5bb-1672-4bc1-a87a-b6eb88eb392c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:59.851208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d374f5bb-1672-4bc1-a87a-b6eb88eb392c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:54:59.852723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d374f5bb-1672-4bc1-a87a-b6eb88eb392c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:54:59.860336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:54:59.899253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f0a74-9584-4888-8d62-1ec364020162 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:54:59.909927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f0a74-9584-4888-8d62-1ec364020162 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:54:59.913440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:54:59.914078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:54:59.914366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:54:59.914648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:54:59.923610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:54:59.925070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88786f6-f0da-4cd3-8bbf-b14d98add247 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:54:59.935068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 738/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:02.298388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf24cd-2b8f-4938-9eeb-49653cbad80e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:02.298953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf24cd-2b8f-4938-9eeb-49653cbad80e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:02.298953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf24cd-2b8f-4938-9eeb-49653cbad80e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:02.300342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:02.306146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f8c6c-abe7-438c-828c-b905352722c4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:02.306549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f8c6c-abe7-438c-828c-b905352722c4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:02.306714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f8c6c-abe7-438c-828c-b905352722c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:02.324420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e58f8c6c-abe7-438c-828c-b905352722c4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:02.325659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e58f8c6c-abe7-438c-828c-b905352722c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:02.333396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:02.372635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf24cd-2b8f-4938-9eeb-49653cbad80e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:02.383621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf24cd-2b8f-4938-9eeb-49653cbad80e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:02.387207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:02.387842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:02.388129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:02.388410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:02.397468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:02.398941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c555d4-0978-48cf-a9b1-bbaffa107330 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:02.408459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 716/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:04.796831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b72c4-d6f6-4378-a95c-754088db1617 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:04.797392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b72c4-d6f6-4378-a95c-754088db1617 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:04.797392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b72c4-d6f6-4378-a95c-754088db1617 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:04.798779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:04.804182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ce63b-83bd-4987-8354-99574a60b8cb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:04.804670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ce63b-83bd-4987-8354-99574a60b8cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:04.804853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ce63b-83bd-4987-8354-99574a60b8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:04.822436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-808ce63b-83bd-4987-8354-99574a60b8cb None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:04.823657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-808ce63b-83bd-4987-8354-99574a60b8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:04.831507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:04.869831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b72c4-d6f6-4378-a95c-754088db1617 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:04.880332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b72c4-d6f6-4378-a95c-754088db1617 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:04.884104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:04.884755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:04.885040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:04.885319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:04.894528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:04.895942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc0dc1-115c-4134-a83e-363f03e2e9d2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:04.905749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 741/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:07.260747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c373e31-79a5-4c44-9113-9721aeb1310e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:07.261361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c373e31-79a5-4c44-9113-9721aeb1310e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:07.261476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c373e31-79a5-4c44-9113-9721aeb1310e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:07.262999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:07.270811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b435d2-3625-4202-9b3b-089f77c93a37 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:07.271250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b435d2-3625-4202-9b3b-089f77c93a37 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:07.271374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b435d2-3625-4202-9b3b-089f77c93a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:07.277689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=70 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:55:07.290698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-00b435d2-3625-4202-9b3b-089f77c93a37 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:07.291893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-00b435d2-3625-4202-9b3b-089f77c93a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:07.299800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:55:07.338544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c373e31-79a5-4c44-9113-9721aeb1310e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:07.344256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=73 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:55:07.350512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c373e31-79a5-4c44-9113-9721aeb1310e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:07.354478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:07.355106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:07.355387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:07.355663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:07.364760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:07.366297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c88111-1984-4271-93c8-b33d52fd29cb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:07.376956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 719/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:09.732997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb444a3-d268-49c5-a262-17caad484e11 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:09.733591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb444a3-d268-49c5-a262-17caad484e11 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:09.733591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb444a3-d268-49c5-a262-17caad484e11 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:09.734936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:09.740148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c338ad-9e91-457f-91c9-1d80093f5002 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:09.740572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c338ad-9e91-457f-91c9-1d80093f5002 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:09.740822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c338ad-9e91-457f-91c9-1d80093f5002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:09.759317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-16c338ad-9e91-457f-91c9-1d80093f5002 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:09.760507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16c338ad-9e91-457f-91c9-1d80093f5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:09.769401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:09.806320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb444a3-d268-49c5-a262-17caad484e11 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:09.817058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb444a3-d268-49c5-a262-17caad484e11 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:09.821416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:09.822045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:09.822331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:09.822668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:09.831532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:09.832994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71b4c1-2d33-483c-b664-2f119e6af4c1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:09.842883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 744/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:12.099254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0cb8-97ac-4af0-995a-035524cc9da1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:12.099788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0cb8-97ac-4af0-995a-035524cc9da1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:12.099895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0cb8-97ac-4af0-995a-035524cc9da1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:12.101421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:12.107068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ac9c8-ddb9-43e2-8de3-8a6fe7045c3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:12.107586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ac9c8-ddb9-43e2-8de3-8a6fe7045c3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:12.107752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ac9c8-ddb9-43e2-8de3-8a6fe7045c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:12.125461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-117ac9c8-ddb9-43e2-8de3-8a6fe7045c3c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:12.126667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-117ac9c8-ddb9-43e2-8de3-8a6fe7045c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:12.135175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:12.171866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0cb8-97ac-4af0-995a-035524cc9da1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:12.182777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0cb8-97ac-4af0-995a-035524cc9da1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:12.186333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:12.186971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:12.187260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:12.187542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:12.197170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:12.198631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92431-9a4e-45d2-ac44-7c34bb4190d2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:12.208359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 722/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:14.441130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd295-560d-4ada-bb00-16436e720101 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:14.441696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd295-560d-4ada-bb00-16436e720101 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:14.441696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd295-560d-4ada-bb00-16436e720101 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:14.443070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:14.448563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572a6b5-f4b0-4834-a47a-1630c7226a53 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:14.448995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572a6b5-f4b0-4834-a47a-1630c7226a53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:14.449163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572a6b5-f4b0-4834-a47a-1630c7226a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:14.466582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2572a6b5-f4b0-4834-a47a-1630c7226a53 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:14.467785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2572a6b5-f4b0-4834-a47a-1630c7226a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:14.475494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:14.513310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd295-560d-4ada-bb00-16436e720101 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:14.533121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd295-560d-4ada-bb00-16436e720101 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:14.536597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:14.537274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:14.537559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:14.537833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:14.546563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:14.547973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4850ca0-0cd4-45af-8d8f-bfe46c16f4e0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:14.557589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 747/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:24.461470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=17 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:55:24.553036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=17 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:55:48.045104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ff0f1-7a33-4faf-90c4-2ef3c7b5585b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:48.045610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ff0f1-7a33-4faf-90c4-2ef3c7b5585b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:48.045722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ff0f1-7a33-4faf-90c4-2ef3c7b5585b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:48.047018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 724/1471] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:48.052065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb559c2-7697-4368-9b7c-87b1cdeee78d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:48.052476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb559c2-7697-4368-9b7c-87b1cdeee78d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:48.052641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb559c2-7697-4368-9b7c-87b1cdeee78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:48.070540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9fb559c2-7697-4368-9b7c-87b1cdeee78d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:48.071720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fb559c2-7697-4368-9b7c-87b1cdeee78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:48.079380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 748/1472] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:48.127069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8071-a662-47f6-b873-e81f37930548 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:48.127362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8071-a662-47f6-b873-e81f37930548 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:48.127644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8071-a662-47f6-b873-e81f37930548 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:48.129037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 725/1473] 10.222.0.61 () {54 vars in 962 bytes} [Thu May 19 21:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:48.134384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d9d1-b8aa-4da3-81cb-a9ea0d9c49e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:48.134676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d9d1-b8aa-4da3-81cb-a9ea0d9c49e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:48.134957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d9d1-b8aa-4da3-81cb-a9ea0d9c49e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:48.153909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3d37d9d1-b8aa-4da3-81cb-a9ea0d9c49e1 None None] MFA Rules not processed for user `de01947b95e746f8a5d6f711d982f97e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:48.155114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3d37d9d1-b8aa-4da3-81cb-a9ea0d9c49e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:48.161554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 749/1474] 10.222.0.61 () {58 vars in 1059 bytes} [Thu May 19 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:48.168830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e8071-a662-47f6-b873-e81f37930548 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:48.179448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e8071-a662-47f6-b873-e81f37930548 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:48.182994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:48.183624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:48.183917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:48.184199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:48.192965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:48.194948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8381225-fb39-4c29-958f-112052af7d7a service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:48.204199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 726/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:50.545651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78716a-6a4d-4ea7-9d80-add3a94aad79 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:50.546304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78716a-6a4d-4ea7-9d80-add3a94aad79 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:50.546304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78716a-6a4d-4ea7-9d80-add3a94aad79 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:50.547597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 750/1476] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:50.554304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465af-ea86-499a-be99-5b3d3e67ddc5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:50.554784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465af-ea86-499a-be99-5b3d3e67ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:50.555150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465af-ea86-499a-be99-5b3d3e67ddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:50.574256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-603465af-ea86-499a-be99-5b3d3e67ddc5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:50.575460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-603465af-ea86-499a-be99-5b3d3e67ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:50.583244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 727/1477] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:50.607190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78716a-6a4d-4ea7-9d80-add3a94aad79 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:50.620117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78716a-6a4d-4ea7-9d80-add3a94aad79 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:50.624573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:50.625243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:50.625534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:50.625815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:50.626555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:50.628304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:55:50.628304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 21:55:50.628304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 21:55:50.629483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a502263-f70c-4d40-93a9-e93c7485442a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:50.649899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 751/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Thu May 19 21:55:50 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:55:52.902763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725ad-6486-4e77-bb12-29743e25be94 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:52.903305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725ad-6486-4e77-bb12-29743e25be94 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:52.903305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725ad-6486-4e77-bb12-29743e25be94 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:52.904702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 728/1479] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:52.909902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6b0f-9121-4667-b799-e971916791f9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:52.910298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6b0f-9121-4667-b799-e971916791f9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:52.910468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6b0f-9121-4667-b799-e971916791f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:52.928860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6d2d6b0f-9121-4667-b799-e971916791f9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:52.930021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d2d6b0f-9121-4667-b799-e971916791f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:52.937957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 752/1480] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:52.983155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d725ad-6486-4e77-bb12-29743e25be94 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:52.997153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d725ad-6486-4e77-bb12-29743e25be94 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:53.002438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:53.003061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:53.003344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:53.003624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:53.011594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:53.013216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e7932-f1d2-4627-847c-9670433f9a83 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:53.023069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 729/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:55.431322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008048-9519-482e-ba63-13c1e8ea0ba5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:55.431888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008048-9519-482e-ba63-13c1e8ea0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:55.431888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008048-9519-482e-ba63-13c1e8ea0ba5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:55.433331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 753/1482] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:55.438668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263714-b0d9-4695-9b25-035485df8b2d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:55.439072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263714-b0d9-4695-9b25-035485df8b2d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:55.439222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263714-b0d9-4695-9b25-035485df8b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:55.457370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1d263714-b0d9-4695-9b25-035485df8b2d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:55.458557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1d263714-b0d9-4695-9b25-035485df8b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:55.467783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 730/1483] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:55.503648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008048-9519-482e-ba63-13c1e8ea0ba5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:55.514761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008048-9519-482e-ba63-13c1e8ea0ba5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:55.518308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:55.518941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:55.519226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:55.519503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:55.528953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:55.530373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307693e-da8c-4413-9c50-a43c39d24fd5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:55.539740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 754/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:55:57.861359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868ac0-2ff7-4922-a61b-33c11a16d90c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:57.861916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868ac0-2ff7-4922-a61b-33c11a16d90c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:57.862013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868ac0-2ff7-4922-a61b-33c11a16d90c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:57.863305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 731/1485] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:55:57.868915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2b22-e6c8-470f-8d92-adcc82cf0c0f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:57.869336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2b22-e6c8-470f-8d92-adcc82cf0c0f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:57.869489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd2b22-e6c8-470f-8d92-adcc82cf0c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:57.887618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cfcd2b22-e6c8-470f-8d92-adcc82cf0c0f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:55:57.888859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfcd2b22-e6c8-470f-8d92-adcc82cf0c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:57.898092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 755/1486] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:55:57.934550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35868ac0-2ff7-4922-a61b-33c11a16d90c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:57.945770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35868ac0-2ff7-4922-a61b-33c11a16d90c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:57.949246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:57.949874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:57.950160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:57.950436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:57.959927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:57.961419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a11118-1207-4b1b-b6f4-f80fe80ec001 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:57.970639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 732/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:00.427128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d081e-8642-41e1-9fc9-dab81b25023a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:00.427778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d081e-8642-41e1-9fc9-dab81b25023a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:00.427778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d081e-8642-41e1-9fc9-dab81b25023a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:00.429128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 756/1488] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:56:00.434304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9422-2729-4f70-b1d3-ca4aacc474da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:00.434752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9422-2729-4f70-b1d3-ca4aacc474da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:00.434855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9422-2729-4f70-b1d3-ca4aacc474da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:00.452855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-734e9422-2729-4f70-b1d3-ca4aacc474da None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:56:00.454045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-734e9422-2729-4f70-b1d3-ca4aacc474da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:56:00.462524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 733/1489] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:56:00.506809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d081e-8642-41e1-9fc9-dab81b25023a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:56:00.517334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d081e-8642-41e1-9fc9-dab81b25023a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:56:00.520779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:56:00.521956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:00.522236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:00.522512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:00.531191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:56:00.532600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6362385-d8a7-47a7-a03b-82a20ed45023 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:56:00.542510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 757/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:02.975756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336acac3-b2fd-4d18-8c5d-412e25f8f937 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:02.976323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336acac3-b2fd-4d18-8c5d-412e25f8f937 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:02.976323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336acac3-b2fd-4d18-8c5d-412e25f8f937 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:02.977983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 734/1491] 10.222.0.61 () {54 vars in 926 bytes} [Thu May 19 21:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:56:02.983214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457e22-dbd6-43da-8bd0-0952f9545314 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:02.983638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457e22-dbd6-43da-8bd0-0952f9545314 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:02.983774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14457e22-dbd6-43da-8bd0-0952f9545314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:03.001469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14457e22-dbd6-43da-8bd0-0952f9545314 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:56:03.002689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14457e22-dbd6-43da-8bd0-0952f9545314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:56:03.010980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 758/1492] 10.222.0.61 () {58 vars in 1023 bytes} [Thu May 19 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:56:03.048841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336acac3-b2fd-4d18-8c5d-412e25f8f937 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:56:03.059624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336acac3-b2fd-4d18-8c5d-412e25f8f937 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:56:03.065381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:56:03.066029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:03.066305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:03.066585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:03.076868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:56:03.078350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117f468-cc8d-4af8-b44b-b3c35756d252 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:56:03.088696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 735/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:12.996414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=33 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:56:13.084122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:46.356289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.356910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.356910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.374777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:57:46.375955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:46.383767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 759/1494] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:57:46.393491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.393491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.394410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.413126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:57:46.414301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:46.422156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 736/1495] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:57:46.426827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.436450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd1988-29f0-4672-b3ef-927372311115 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.441870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.442520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.442807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.443095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.443828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:57:46.444140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.445182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.445182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 21:57:46.445182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 21:57:46.446167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.458267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 760/1496] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 21:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:57:46.466348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.475957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb891b-1423-4637-8a05-9d344dbf1032 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.480758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.481400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.481684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.481957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.482650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.484462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.484462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 21:57:46.484462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 21:57:46.485695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.523868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 737/1497] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:57:46.528135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.538418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514fab7-204b-4cf0-9e2b-762dca6de09a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.594496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.595153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.595467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.595750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.596476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.597521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.597521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 21:57:46.597521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 21:57:46.598727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.618607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 761/1498] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:57:46.622801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.634543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61546341-b025-44d5-a678-508e574d94f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.638967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.639621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.639902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.640177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.640935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.641954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.641954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 21:57:46.641954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 21:57:46.643466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.655986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 738/1499] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 21:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:57:46.660698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.670448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d81f-a3ee-4972-8615-fc39b939458c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.675775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.676422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.676750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.677038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.677770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.678775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.678775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 21:57:46.678775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 21:57:46.679658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c3b5-e38a-4881-8dd7-b9f36df4db71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.693384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 762/1500] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 21:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:57:46.697374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.707136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394d53-61e2-4c61-9607-4ed26cde19b4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.711552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.712412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.712728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.713008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] PATH_INFO: `/v3/projects/78afb15b7d17470395b74705ddbc26bc/users/e5c30df0fe854d14a5976b9add9a6a0a/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.728087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78afb15b7d17470395b74705ddbc26bc, user_id=e5c30df0fe854d14a5976b9add9a6a0a, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.729169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:46.729169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 21:57:46.729169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 21:57:46.731084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c22a9-8def-4524-ba89-a9f9130b774f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.744609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 739/1501] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 21:57:46 2022] PUT /identity/v3/projects/78afb15b7d17470395b74705ddbc26bc/users/e5c30df0fe854d14a5976b9add9a6a0a/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:46.756288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499aee-8350-4ad3-8ffc-b6f7b5b3d78c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.756581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499aee-8350-4ad3-8ffc-b6f7b5b3d78c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.756892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499aee-8350-4ad3-8ffc-b6f7b5b3d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.784577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd499aee-8350-4ad3-8ffc-b6f7b5b3d78c None None] MFA Rules not processed for user `e5c30df0fe854d14a5976b9add9a6a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:57:46.786161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd499aee-8350-4ad3-8ffc-b6f7b5b3d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:46.849512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 763/1502] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:57:46.864408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0307-2524-4ac7-a801-33851b2d5fe6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.864727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0307-2524-4ac7-a801-33851b2d5fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.865020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0307-2524-4ac7-a801-33851b2d5fe6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.866407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 740/1503] 10.222.0.61 () {54 vars in 963 bytes} [Thu May 19 21:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:57:46.871356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9989-0741-4487-aa6b-e0467cb9ee52 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.871600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9989-0741-4487-aa6b-e0467cb9ee52 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.871889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9989-0741-4487-aa6b-e0467cb9ee52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.891146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-21ba9989-0741-4487-aa6b-e0467cb9ee52 None None] MFA Rules not processed for user `41cfa205823a4c29abea3001bf086711`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:57:46.892380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21ba9989-0741-4487-aa6b-e0467cb9ee52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:46.935654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:46.944512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 764/1504] 10.222.0.61 () {58 vars in 1060 bytes} [Thu May 19 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:57:46.952022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd0307-2524-4ac7-a801-33851b2d5fe6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:46.962496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd0307-2524-4ac7-a801-33851b2d5fe6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:46.965952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:46.966571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:46.966853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:46.967127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:46.975285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:46.976760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:46.985518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 741/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:57:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:47.027912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a2fa-fbc7-4210-a4fb-90a5e070ced9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.028202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a2fa-fbc7-4210-a4fb-90a5e070ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.028484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a2fa-fbc7-4210-a4fb-90a5e070ced9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.029913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 765/1506] 10.222.0.61 () {52 vars in 829 bytes} [Thu May 19 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:57:47.033536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:47.094520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ca82-b551-4df6-9b00-d40cc2add02f service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:47.099163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:47.100028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.100316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.100588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] PATH_INFO: `/v3/users/e5c30df0fe854d14a5976b9add9a6a0a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.101802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c30df0fe854d14a5976b9add9a6a0a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:47.103088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:47.103088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 21:57:47.103088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 21:57:47.104288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:47.184300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c30df0fe854d14a5976b9add9a6a0a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.197258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c30df0fe854d14a5976b9add9a6a0a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.197954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c30df0fe854d14a5976b9add9a6a0a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.198685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c30df0fe854d14a5976b9add9a6a0a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.216937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c30df0fe854d14a5976b9add9a6a0a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.221425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:47.233451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 742/1507] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 21:57:47 2022] DELETE /identity/v3/users/e5c30df0fe854d14a5976b9add9a6a0a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:47.239462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.239755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.240042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.257268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:57:47.258445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:47.319495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 766/1508] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:57:47.328956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:47.331037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:47.388116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd63c90-ba6b-4557-ab40-3733b2a2c204 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:47.391711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:47.392342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.392748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.393043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.401801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:47.403250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf71cf1-7eca-4934-9aea-de30a7013edb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:47.412900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 743/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:47.760886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:47.764644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:47.776137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535864c6-7c58-4edd-9e75-a5caae313068 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:47.781072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:47.781962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.782365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.782647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] PATH_INFO: `/v3/projects/78afb15b7d17470395b74705ddbc26bc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.784793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=78afb15b7d17470395b74705ddbc26bc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:47.785825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:57:47.785825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 21:57:47.785825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 21:57:47.787071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:47.848890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78afb15b7d17470395b74705ddbc26bc', 'request_id': 'req-47464cc9-8591-42e8-a759-b414f2107aab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.862835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78afb15b7d17470395b74705ddbc26bc', 'request_id': 'req-47464cc9-8591-42e8-a759-b414f2107aab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.873046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78afb15b7d17470395b74705ddbc26bc', 'request_id': 'req-47464cc9-8591-42e8-a759-b414f2107aab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.884470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] The token cache is being invalidate because project 78afb15b7d17470395b74705ddbc26bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:57:47.884794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.885518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-47464cc9-8591-42e8-a759-b414f2107aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:47.887059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 767/1510] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 21:57:47 2022] DELETE /identity/v3/projects/78afb15b7d17470395b74705ddbc26bc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:57.408883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:57.881540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:59:30.716553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:30.717206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:30.717206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:30.719006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:30.723928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fffaa-c48f-4896-91a5-2b60ffe06c5f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:30.724358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fffaa-c48f-4896-91a5-2b60ffe06c5f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:30.724509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fffaa-c48f-4896-91a5-2b60ffe06c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:30.742923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f54fffaa-c48f-4896-91a5-2b60ffe06c5f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:30.744119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f54fffaa-c48f-4896-91a5-2b60ffe06c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:30.803714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:59:30.979007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:30.980832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:31.061783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c2232-b766-4212-a6f2-65453c9bfc16 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:31.067358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:31.068017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:31.068306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:31.068589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:31.077021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:31.078635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a08ab8-9ff3-434f-8291-b260fcc38531 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:31.087937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 745/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:59:31.715155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f922-292d-404b-b814-495a9799f3d1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:31.715362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f922-292d-404b-b814-495a9799f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:31.715631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f922-292d-404b-b814-495a9799f3d1 None None] PATH_INFO: `/v3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:31.717010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 769/1514] 10.222.0.49 () {54 vars in 925 bytes} [Thu May 19 21:59:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:59:31.742640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7f2e-988c-44e3-b481-342d8926233f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:31.743182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7f2e-988c-44e3-b481-342d8926233f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:31.743275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7f2e-988c-44e3-b481-342d8926233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:31.763432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-00eb7f2e-988c-44e3-b481-342d8926233f None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:31.765725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00eb7f2e-988c-44e3-b481-342d8926233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:31.843901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 746/1515] 10.222.0.49 () {58 vars in 1015 bytes} [Thu May 19 21:59:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:32.036839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d505f0-be6e-4d54-8ae3-ae8b4ade7917 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:32.037309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d505f0-be6e-4d54-8ae3-ae8b4ade7917 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:32.037420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d505f0-be6e-4d54-8ae3-ae8b4ade7917 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:32.038760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 770/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:32.043674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99690-3818-42b4-874b-c7280472e15d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:32.043971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99690-3818-42b4-874b-c7280472e15d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:32.044262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99690-3818-42b4-874b-c7280472e15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:32.062395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-44f99690-3818-42b4-874b-c7280472e15d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:32.063778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44f99690-3818-42b4-874b-c7280472e15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:32.071670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 747/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:32.224520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d505f0-be6e-4d54-8ae3-ae8b4ade7917 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:32.239120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d505f0-be6e-4d54-8ae3-ae8b4ade7917 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:32.244433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:32.245086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:32.245370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:32.245645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:32.254359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:32.255949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b8897-0da8-433b-879e-79e0e2c698ee service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:32.265808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 771/1518] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:59:35.726874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d67f15-86f7-40d1-a033-2d0f5166802a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:35.727441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d67f15-86f7-40d1-a033-2d0f5166802a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:35.727441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d67f15-86f7-40d1-a033-2d0f5166802a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:35.728822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 748/1519] 10.222.0.61 () {54 vars in 960 bytes} [Thu May 19 21:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:35.734246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee557-2412-477e-abc3-24033d32903d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:35.734711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee557-2412-477e-abc3-24033d32903d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:35.734822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee557-2412-477e-abc3-24033d32903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:35.753052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-41cee557-2412-477e-abc3-24033d32903d None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:35.754285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-41cee557-2412-477e-abc3-24033d32903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:35.762765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 772/1520] 10.222.0.61 () {58 vars in 1057 bytes} [Thu May 19 21:59:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:35.770824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d67f15-86f7-40d1-a033-2d0f5166802a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:35.786197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d67f15-86f7-40d1-a033-2d0f5166802a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:35.791246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:35.791886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:35.792172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:35.792454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:35.800852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:35.802475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9205d6b-2aef-498e-9e3d-48d1d123064d service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:35.813546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 749/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:59:43.575586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64b0f-5f76-4ab9-861f-ef573b34685e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:43.576718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64b0f-5f76-4ab9-861f-ef573b34685e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:43.576854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64b0f-5f76-4ab9-861f-ef573b34685e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:43.578190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:43.583068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dd07b-397f-4bc8-8dd5-ec079268c775 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:43.583474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dd07b-397f-4bc8-8dd5-ec079268c775 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:43.583634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dd07b-397f-4bc8-8dd5-ec079268c775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:43.602003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1e7dd07b-397f-4bc8-8dd5-ec079268c775 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:43.603211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e7dd07b-397f-4bc8-8dd5-ec079268c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:43.611368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:43.767950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64b0f-5f76-4ab9-861f-ef573b34685e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:43.782150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64b0f-5f76-4ab9-861f-ef573b34685e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:43.788405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:43.789133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:43.789411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:43.789683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:43.798400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:43.800008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2e12-a3a8-4e37-804a-695ff7cdd76e service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:43.809267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 774/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:59:44.708773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd281ec-7c85-48fb-88b6-cd2e00612c7d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:44.709339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd281ec-7c85-48fb-88b6-cd2e00612c7d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:44.709339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd281ec-7c85-48fb-88b6-cd2e00612c7d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:44.710586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:59:44.715479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8807e-f0ea-4bd6-813f-d0d6582fff9d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:44.715879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8807e-f0ea-4bd6-813f-d0d6582fff9d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:44.716077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8807e-f0ea-4bd6-813f-d0d6582fff9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:44.734962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-96d8807e-f0ea-4bd6-813f-d0d6582fff9d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:44.736265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96d8807e-f0ea-4bd6-813f-d0d6582fff9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:44.744433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:44.894921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd281ec-7c85-48fb-88b6-cd2e00612c7d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:44.910162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd281ec-7c85-48fb-88b6-cd2e00612c7d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:44.915647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:44.916342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:44.916662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:44.917224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:44.925748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:44.927348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ad27b-9d17-4a28-a77f-406ac1cc63e9 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:44.937463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 752/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:59:48.963281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:48.963823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:48.963823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:48.965290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:48.970870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8756d-4da6-44a8-8be8-4fe10c8fba59 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:48.971271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8756d-4da6-44a8-8be8-4fe10c8fba59 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:48.971439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8756d-4da6-44a8-8be8-4fe10c8fba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:48.990114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63e8756d-4da6-44a8-8be8-4fe10c8fba59 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:48.991329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63e8756d-4da6-44a8-8be8-4fe10c8fba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:48.999195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:49.036424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:49.038174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:49.094374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246d741-403e-4e35-bcd0-84660a011225 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:49.098122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:49.098906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:49.099269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:49.099552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:49.109315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:49.110757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c3275-276e-4313-a305-7439393d3628 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:49.120411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 777/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:59:51.362474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679591e-2fae-4dc2-876b-fefd4af83a9f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:51.362474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679591e-2fae-4dc2-876b-fefd4af83a9f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:51.362474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679591e-2fae-4dc2-876b-fefd4af83a9f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:51.363885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:51.369270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f96901-1efd-4950-b13f-ed35eb15cfe7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:51.369698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f96901-1efd-4950-b13f-ed35eb15cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:51.369829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f96901-1efd-4950-b13f-ed35eb15cfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:51.389083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-08f96901-1efd-4950-b13f-ed35eb15cfe7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:51.390282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-08f96901-1efd-4950-b13f-ed35eb15cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:51.397981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:51.433242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679591e-2fae-4dc2-876b-fefd4af83a9f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:51.443903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679591e-2fae-4dc2-876b-fefd4af83a9f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:51.447433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:51.448054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:51.448335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:51.448605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:51.457827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:51.459258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449819-ae63-45a4-903d-c9785fbc5c9a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:51.469321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 755/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:59:55.296006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:55.296534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:55.296534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:55.297956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:55.303098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221154-28cd-4d37-9ea4-6324981524ac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:55.303503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221154-28cd-4d37-9ea4-6324981524ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:55.303667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221154-28cd-4d37-9ea4-6324981524ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:55.321842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fc221154-28cd-4d37-9ea4-6324981524ac None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:55.323042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc221154-28cd-4d37-9ea4-6324981524ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:55.331241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:55.603014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:55.604701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:55.661774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84bddd-86a1-4a4c-b1b4-8e02bba5721f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:55.665295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:55.665932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:55.666213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:55.666489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:55.675161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:55.676578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ad1e-f141-47c6-aef0-3a196684e9c7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:55.681390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:59:55.686970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 780/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:59:57.975290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:57.975843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:57.975843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:57.977502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:59:57.982648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56653dea-65f9-442e-952c-041cb7f97d16 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:57.983768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56653dea-65f9-442e-952c-041cb7f97d16 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:57.983874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56653dea-65f9-442e-952c-041cb7f97d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:58.001544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-56653dea-65f9-442e-952c-041cb7f97d16 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:59:58.002740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-56653dea-65f9-442e-952c-041cb7f97d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:58.010398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:59:58.283197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:59:58.284901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:59:58.292371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:59:58.299124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70d9d-626d-4389-925b-a9769c04b543 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:59:58.302683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:59:58.303353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:59:58.303641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:59:58.303921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:59:58.313401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:59:58.314884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf1016-f459-4a9c-9d76-fec25389f6f0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:59:58.324215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 758/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 21:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:00:07.996286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:00:08.320109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:30.114763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9a42-e0d0-46fa-b342-cd336f5bf263 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.115842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9a42-e0d0-46fa-b342-cd336f5bf263 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.116348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9a42-e0d0-46fa-b342-cd336f5bf263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.143222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-180e9a42-e0d0-46fa-b342-cd336f5bf263 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.149036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-180e9a42-e0d0-46fa-b342-cd336f5bf263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.159038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 782/1540] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.175560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794769-6018-4261-8f10-7854f4fd2347 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.176434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7efc-e528-4b72-9978-c32a8a15f481 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.176758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7efc-e528-4b72-9978-c32a8a15f481 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.177046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7efc-e528-4b72-9978-c32a8a15f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.177449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794769-6018-4261-8f10-7854f4fd2347 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.178107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0794769-6018-4261-8f10-7854f4fd2347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.212648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b07f7efc-e528-4b72-9978-c32a8a15f481 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.212648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b07f7efc-e528-4b72-9978-c32a8a15f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.219565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f0794769-6018-4261-8f10-7854f4fd2347 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.220869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0794769-6018-4261-8f10-7854f4fd2347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.231910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 783/1541] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:01:30.233461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.233902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.234331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.242921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 759/1542] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.244167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dc51-17d8-442a-be78-bfae22ad9b89 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.244462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dc51-17d8-442a-be78-bfae22ad9b89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.244778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dc51-17d8-442a-be78-bfae22ad9b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.273970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.275569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.285715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7ae7dc51-17d8-442a-be78-bfae22ad9b89 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.285715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7ae7dc51-17d8-442a-be78-bfae22ad9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.293879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 784/1543] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.295112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.307519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be4e35-5d00-4047-b9c8-ceb21ddaa421 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.313164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.314739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.315211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.315659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.319947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 760/1544] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.319947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763715f-e5da-4132-bd19-e69ff4751af5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.319947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763715f-e5da-4132-bd19-e69ff4751af5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.319947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763715f-e5da-4132-bd19-e69ff4751af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.320900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.321471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.323311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfacc51-8112-4113-9fb6-62b3396cacf2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.344255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 785/1545] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.345926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74369a89-01a7-47c5-ac56-51c4336c3a00 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.346407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74369a89-01a7-47c5-ac56-51c4336c3a00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.347341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74369a89-01a7-47c5-ac56-51c4336c3a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.356688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d763715f-e5da-4132-bd19-e69ff4751af5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.357829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d763715f-e5da-4132-bd19-e69ff4751af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.365927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 761/1546] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.367181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483ec8-8e04-4f98-87c9-12c4f4aa04b6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.367473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483ec8-8e04-4f98-87c9-12c4f4aa04b6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.367756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483ec8-8e04-4f98-87c9-12c4f4aa04b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.373335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-74369a89-01a7-47c5-ac56-51c4336c3a00 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.374748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-74369a89-01a7-47c5-ac56-51c4336c3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.383572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 786/1547] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.385093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.386046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.386279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.393491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f7483ec8-8e04-4f98-87c9-12c4f4aa04b6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.395091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f7483ec8-8e04-4f98-87c9-12c4f4aa04b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.404300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 762/1548] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.405775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e15203-73b1-48c1-a64c-ad1bd9f73175 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.405983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.406401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e15203-73b1-48c1-a64c-ad1bd9f73175 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.406846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e15203-73b1-48c1-a64c-ad1bd9f73175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.407477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.416986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 787/1549] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.417773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.426638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82e15203-73b1-48c1-a64c-ad1bd9f73175 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.428191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82e15203-73b1-48c1-a64c-ad1bd9f73175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.429528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0ea7f-ea18-4bdc-9399-8d01809dcd2d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.434438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.435103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.435388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.435679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.436379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.437633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:30.437633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 22:01:30.437633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 22:01:30.439074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.439666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 763/1550] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.440990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea3da4-2e28-4ab3-bda4-5465baeb6b24 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.441285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea3da4-2e28-4ab3-bda4-5465baeb6b24 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.441567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea3da4-2e28-4ab3-bda4-5465baeb6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.462052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63ea3da4-2e28-4ab3-bda4-5465baeb6b24 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.463249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63ea3da4-2e28-4ab3-bda4-5465baeb6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.467105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 788/1551] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:30.467875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.524090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 764/1552] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.525597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2554b-2a4b-454a-ba6f-1be25b22bdab None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.525897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2554b-2a4b-454a-ba6f-1be25b22bdab None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.526644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2554b-2a4b-454a-ba6f-1be25b22bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.531035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167e96-9f15-4a4f-8322-d32de4c9966e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.536968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.537609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.537899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.538182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.538923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.539239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.540892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.545943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fdc2554b-2a4b-454a-ba6f-1be25b22bdab None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.547149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fdc2554b-2a4b-454a-ba6f-1be25b22bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.552556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 789/1553] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.553376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.558560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 765/1554] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.558560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3de1-bac2-4b23-a9c3-4d3a8779c96b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.558560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3de1-bac2-4b23-a9c3-4d3a8779c96b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.558560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f3de1-bac2-4b23-a9c3-4d3a8779c96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.564902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546f5cc-c334-438d-bab8-a5a7507642aa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.569476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.570124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.570396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.570673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.571392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.571696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.573330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.577081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-928f3de1-bac2-4b23-a9c3-4d3a8779c96b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.578276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-928f3de1-bac2-4b23-a9c3-4d3a8779c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.590916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 766/1556] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.590916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.590916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.590916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.591353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 790/1556] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.591353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.601643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba81cda-39ec-4b39-8796-d6f2ccfe7953 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.606204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.606853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.607182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.607464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.608218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.608522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.608660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.609759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.610122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89476bbb-8c6f-4e4a-892e-2cba4a8a874f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.617612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 767/1557] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.618379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.623082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 791/1558] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.624440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.624763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.625045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.633542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff92c5-3927-4af0-9009-787735d7e374 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.638848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.639526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.639816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.640096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.640836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.641888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:30.641888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 22:01:30.641888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 22:01:30.643123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.648382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.649682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.659145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 792/1559] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.660090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.668706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 768/1560] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/users => generated 288 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:30.670118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.673186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483af5-7d7f-49a1-835a-dd312bfe9bf8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.677729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.678377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.678658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.678932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.679654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.679773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead221-fc52-4acc-9fd0-3cd28d4115af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.679956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.681561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.685308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.685970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.686256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.686535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.687221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.689267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.693182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 793/1561] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.694043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.703958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778fff3-0a10-4bf7-a3d9-59f9821a89c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.708430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.709102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.709386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.709659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.710370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.710674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.712231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a626ec-2135-43c7-85ea-68775f589b6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.726993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 769/1562] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:30.726993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.727260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 794/1563] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.727260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.727416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.727641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.749233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:30.750435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:30.770484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:30.798008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e658d77-2703-43cd-8a5a-e397098bb43e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.802923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.803577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.803859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.804135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.804855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.806765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.813718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 795/1564] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:30.814545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.821548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:30.824531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435161ee-a16f-49cf-9135-c0276658a3d3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.830130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.830791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.831206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.831494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.832187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.834165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.837917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 770/1565] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:30.837917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.864833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 796/1566] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:30.864833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.902160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127cd2-0f4b-4bf7-9182-3bf3c1f0e82a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.934865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e65c5-d975-49b3-84ef-5a7fb28db65c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.939779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.940454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.940778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.941057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.941795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.943415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.954026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 797/1567] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:30.954883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.966555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2c93-33dc-41b4-9030-98d8af8c1c94 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:30.968980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.969765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.970097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.970314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.971045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:30.971350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.971456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:30.980732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:30.980732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:30.980969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:30.980969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 22:01:30.980969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 22:01:30.981165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:30.981165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:30.981348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.981445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:30.985587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 771/1568] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:30.986380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:30.995789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da6d06-dc5c-4ca5-b94d-56ed8f87e71c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.000331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.001197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.001491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.001766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.002504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.004503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.013680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 798/1569] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.014565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.031573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 772/1570] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:30 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.031933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.041439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1f0e5-09e2-4005-9d16-4357f6cf01b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.078899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a38b4-74b4-482d-ac33-7f54fea2d5a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.083552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.084216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.084506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.084822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.085544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.087459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.097975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.098648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.098939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.099222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.099957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:31.100269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.102456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.113588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 799/1571] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.114734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.115464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 773/1572] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:31.116262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.120870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.129622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4af1e-9d91-4ae9-b565-1ad212535ab8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.138085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.182490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17ad8-ffa8-4120-98e9-1e53bc699796 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.187228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.187894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.188033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.188518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.188518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.188736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.189074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.189187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.189352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.190105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.191105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.192066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.212192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 800/1573] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.213063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.217914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 774/1574] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.217914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.223179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6581-af40-4064-821e-fbeb77560db0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.227922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc43f-cce9-4e3d-99fa-76c43387048d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.228061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.228677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.228969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.229249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.229969 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.231779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.232804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.233655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.234309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.234624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.235318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.237300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.241853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 801/1575] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.245476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.255133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08976053-8a7c-4c8a-9fb9-edeb601b0bcc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.260556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.261247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.261537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.262407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.263383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.264888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 775/1576] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.265328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.268945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.286862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 802/1577] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.287686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.297529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e83d-becf-4639-add4-84a12ad660f0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.335863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35216c4-7e14-4007-8718-ed02b69a118c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.341231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.341892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.342175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.342477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.343210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.344873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.354925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 776/1578] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.358221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.358369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.358467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.358467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.358467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.358467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.359441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.364925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e23a-0ac5-43ad-b90c-7a0db5bcef66 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.369718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.370350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.370653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.370912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.372085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.374021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.380598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 803/1579] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.384724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.395704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8cdfc-17c5-494f-95f7-a9f8b3651511 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.400401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.401091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.401401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.401665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.405095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 777/1580] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.405246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.405342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.405453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.417903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 804/1581] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.417903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.425799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae8c9-5967-432b-86cf-ec1969d08cd3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.459533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.468546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.471557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7ae5-0409-42d5-9198-db79dbd4bde1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.477350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.478023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.478313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.478595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.479357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.481595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.486543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.487429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.487711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.487986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] PATH_INFO: `/v3/projects/26de7fe4dc7c417f9f7a3344e6d049d3/users/758eb207b7214f0eb4b14022a1ea46f2/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.491632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 778/1582] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.493447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.503406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9128-3d90-419e-ba71-7a657c66015f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.504041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26de7fe4dc7c417f9f7a3344e6d049d3, user_id=758eb207b7214f0eb4b14022a1ea46f2, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.505125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:31.505125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 22:01:31.505125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 22:01:31.507050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.507971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.508643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.508925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.509198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.509879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.512061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.522143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 805/1583] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:31 2022] PUT /identity/v3/projects/26de7fe4dc7c417f9f7a3344e6d049d3/users/758eb207b7214f0eb4b14022a1ea46f2/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:31.533960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.541100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 779/1584] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.545708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.555173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a3a8-4e4a-42f7-94c6-80352411725b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.588510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed96c0-2cf3-4bcf-a173-a9c7796a3102 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.594281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.594954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.595243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.595522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.596257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.597932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.608950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 806/1585] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.609876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.615434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.616110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.616397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.616713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.617444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.619064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.621882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c339fb4-f924-418a-8158-6ed0b0345a10 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.627066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.627802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.628093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.628401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.629191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.632592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 780/1586] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.632592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.632845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.640026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4af95-1b7d-4552-bc43-527e79f87bec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.641516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 807/1587] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.642315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.648125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.648837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.649153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.649446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.650145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.652082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265b27a-3bc5-41ea-9b3c-c95e933d9099 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.652998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9c18-d664-48c9-82c1-eb7b7b026aef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.657654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.658317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.658598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.658874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.659587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.661201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.673421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 808/1588] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.674178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 781/1589] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:31.675451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.675722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.676002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.677105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.687509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ece8-62d6-40f4-b6da-e96d97634257 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.692257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.692952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.693242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.693551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.694747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.696907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.704385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] MFA Rules not processed for user `758eb207b7214f0eb4b14022a1ea46f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:31.705643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:31.708640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 809/1590] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.709471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.719897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67ab4c-fa58-4394-bfb7-c3cc4261a636 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.724463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.725138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.725427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.725706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.726682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.728253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.738496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 810/1591] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.739279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.750455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc08fcd-b11c-4e33-b9a8-4c802c7a4f70 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.755127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.755819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.756111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.756395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.757155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.759012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.770401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 811/1592] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.771567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.778964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 782/1593] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:31.780129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.784499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf399c-652a-4280-9c16-d6f8c22dc417 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.789291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.790175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.790458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.790734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] PATH_INFO: `/v3/projects/996e01242d5049279cd562458f77562d/users/7ef029bafd694b46973cdff0c87123e2/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.806949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5576c9-e493-4260-8902-228a4146ad78 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.809415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996e01242d5049279cd562458f77562d, user_id=7ef029bafd694b46973cdff0c87123e2, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.812234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.813250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.813944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.814330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.814710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.815549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.817961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.829907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 783/1594] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.833947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 812/1595] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:31 2022] PUT /identity/v3/projects/996e01242d5049279cd562458f77562d/users/7ef029bafd694b46973cdff0c87123e2/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:31.834096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.834210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.844643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dda44-84c4-44b3-8d0b-b58517adcf1f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.871347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.880365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:31.898825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f5cb5-b40c-4b1e-abf6-274437337f35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.904450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.905390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.905609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.905751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.905926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] PATH_INFO: `/v3/projects/12d00f7cc000446d8c2f294215d691e5/users/8e78bf605b25422da6f29b5dc95c0b6f/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.906285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.906574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.906855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.907592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.909258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.919304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 784/1596] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:31.920100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.925114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d00f7cc000446d8c2f294215d691e5, user_id=8e78bf605b25422da6f29b5dc95c0b6f, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.927756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.929502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748af7e6-b65f-4d4f-94c1-2fc949997f0a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:31.935558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:31.936709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:31.937011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:31.937273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] PATH_INFO: `/v3/projects/bb177f0cfe3a432c80d389c2e048e82e/users/214c6759db6c466cb132fefd2ddea57b/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:31.941635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 813/1597] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:31 2022] PUT /identity/v3/projects/12d00f7cc000446d8c2f294215d691e5/users/8e78bf605b25422da6f29b5dc95c0b6f/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:31.945470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:31.953849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb177f0cfe3a432c80d389c2e048e82e, user_id=214c6759db6c466cb132fefd2ddea57b, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:31.956550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:31.971425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 785/1598] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:31 2022] PUT /identity/v3/projects/bb177f0cfe3a432c80d389c2e048e82e/users/214c6759db6c466cb132fefd2ddea57b/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:31.972212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.012463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148db7d-4cfb-4c39-b092-f0a961d7b80a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.047259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cc31d-4282-43b6-9dc9-f9a5b35d3cf4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.051944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.052912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.053216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.053493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] PATH_INFO: `/v3/projects/f9b4725e31b24d4fbdcbf849cfbb7f45/users/383ec9ad9bac4dcfb6cf6fc68a0db3b0/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.067938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b4725e31b24d4fbdcbf849cfbb7f45, user_id=383ec9ad9bac4dcfb6cf6fc68a0db3b0, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.070639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b67fad-ab52-490e-b261-4b10bc48c595 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.072603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.073297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.073590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.073873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.075311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.077294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.085586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 786/1599] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:31 2022] PUT /identity/v3/projects/f9b4725e31b24d4fbdcbf849cfbb7f45/users/383ec9ad9bac4dcfb6cf6fc68a0db3b0/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:32.086966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9a25-eff3-4356-ab0f-3fb0043207d9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.087260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9a25-eff3-4356-ab0f-3fb0043207d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.087541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9a25-eff3-4356-ab0f-3fb0043207d9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.091186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 814/1600] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:32.091186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.091434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 787/1601] 10.222.0.61 () {54 vars in 961 bytes} [Thu May 19 22:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:32.091434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.091434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.091434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.102035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5202cd-512c-42e5-89db-4d1a5181d105 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.108919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.110126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.158885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.159803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.160084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.160361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] PATH_INFO: `/v3/projects/4a9d5bf488674c00a7c1ed3e678ce29c/users/6ae4e8e69a424bed9a271d567a8b65ec/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.171400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 788/1602] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.172173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.178160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9d5bf488674c00a7c1ed3e678ce29c, user_id=6ae4e8e69a424bed9a271d567a8b65ec, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.181614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c777d-e347-4b5a-bb2e-69ee79176d83 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.182560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52433813-80ab-4994-b51d-9dc6b8fad7ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.184850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.188155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.188910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.189198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.189499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.190237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.191850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fc15f-2a77-4857-aa67-89a515a6cb92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.197328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 815/1603] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:32 2022] PUT /identity/v3/projects/4a9d5bf488674c00a7c1ed3e678ce29c/users/6ae4e8e69a424bed9a271d567a8b65ec/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:32.208082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647cd8-2ffa-4a2b-8485-6d1fc489d9d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.208082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647cd8-2ffa-4a2b-8485-6d1fc489d9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.208082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647cd8-2ffa-4a2b-8485-6d1fc489d9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.208378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 789/1604] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:32.208378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.208378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.208378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.230627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-20647cd8-2ffa-4a2b-8485-6d1fc489d9d6 None None] MFA Rules not processed for user `8e78bf605b25422da6f29b5dc95c0b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.231679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20647cd8-2ffa-4a2b-8485-6d1fc489d9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.234113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.237267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] MFA Rules not processed for user `214c6759db6c466cb132fefd2ddea57b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.239087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.310727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 816/1605] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.312042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136fcf3-ef21-4fce-8718-f33cac18b4f0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.312340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136fcf3-ef21-4fce-8718-f33cac18b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.312750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136fcf3-ef21-4fce-8718-f33cac18b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.331179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e136fcf3-ef21-4fce-8718-f33cac18b4f0 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.332510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e136fcf3-ef21-4fce-8718-f33cac18b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.345839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 790/1606] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.347169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.400649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 817/1607] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.401942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.402784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.403113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.420269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd17f2-ea73-45bc-ad03-5f918da4ae52 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.422983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.424185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.424939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.425829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.426116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.426391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] PATH_INFO: `/v3/projects/2308cb6b98094f2f8fc03c7f13c2d8b0/users/f07074cdff9148b581a33f12749d0277/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.441602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2308cb6b98094f2f8fc03c7f13c2d8b0, user_id=f07074cdff9148b581a33f12749d0277, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.444344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.458220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 791/1608] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:32 2022] PUT /identity/v3/projects/2308cb6b98094f2f8fc03c7f13c2d8b0/users/f07074cdff9148b581a33f12749d0277/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:32.459083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.461108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.495159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 818/1609] 10.222.0.61 () {58 vars in 1058 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.496093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.506683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ddf61-9c0d-4e08-92e0-1b88de1740ce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.519685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.520789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6afb35-b692-4fe0-99aa-43760a0342a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.524444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.525122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.525406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.525680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.566659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.567894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.568865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.569188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.569473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] PATH_INFO: `/v3/projects/2add5497070940cd8e911a8e6274c73c/users/980d1f84ba3549e3b79ddbff8e090538/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.585547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2add5497070940cd8e911a8e6274c73c, user_id=980d1f84ba3549e3b79ddbff8e090538, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.588228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc5475-3474-4a6c-9262-d1528d326e97 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.604422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 819/1610] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:32 2022] PUT /identity/v3/projects/2add5497070940cd8e911a8e6274c73c/users/980d1f84ba3549e3b79ddbff8e090538/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:32.606273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6bed-3442-4641-8007-a0043d4a19b7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.606803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6bed-3442-4641-8007-a0043d4a19b7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.606803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb6bed-3442-4641-8007-a0043d4a19b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.609279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.610857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.622561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 792/1611] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:32.623319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.632979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dae0d1-71b6-4262-8eb1-494260a5e457 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.633606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-12bb6bed-3442-4641-8007-a0043d4a19b7 None None] MFA Rules not processed for user `6ae4e8e69a424bed9a271d567a8b65ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.635142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-12bb6bed-3442-4641-8007-a0043d4a19b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.703803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.704470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.704803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.705083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.705783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.707731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1590-cfc6-410d-affe-7d4c3b8bb04b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.722537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 820/1612] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.723933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858c36-3e38-4782-9aae-fa50814f0c03 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.724225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858c36-3e38-4782-9aae-fa50814f0c03 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.724502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858c36-3e38-4782-9aae-fa50814f0c03 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.726415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 821/1613] 10.222.0.61 () {54 vars in 961 bytes} [Thu May 19 22:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:32.726931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15858c36-3e38-4782-9aae-fa50814f0c03 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.742442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 793/1614] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:32.744572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.745122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.745637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.768335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.769930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.798871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15858c36-3e38-4782-9aae-fa50814f0c03 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.842152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 794/1615] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.843113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:32.863000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.874388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.877169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.877448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.877746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.903239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:32.903239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedaa70-0d26-4be9-a7b1-419dee29de96 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:32.912424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 822/1616] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:32.913896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad795bc-ce27-4012-8899-fc30515333af None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.914263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad795bc-ce27-4012-8899-fc30515333af None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.914619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad795bc-ce27-4012-8899-fc30515333af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.920426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:32.938741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fad795bc-ce27-4012-8899-fc30515333af None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:32.940206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fad795bc-ce27-4012-8899-fc30515333af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:32.941025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124e0fb-b306-4093-8261-c8d5da5c7bc8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:32.948410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:32.949906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.950316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.950673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:32.961545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 823/1617] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:32.963815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:32.964304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:32.964890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.005295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:33.007426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:33.031397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 824/1618] 10.222.0.61 () {58 vars in 1058 bytes} [Thu May 19 22:01:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:33.032430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.049279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a038de-fa80-429f-b03f-15d4cdbc486c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.057577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.057812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.058240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.059338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.060487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.063457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.095166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 825/1619] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:33 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:33.096289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.096484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.099127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.110582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde0a25-6c08-402c-93aa-e9ec6d9d1c1b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.115170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.115958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.116452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.116990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.130454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.132471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.148832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 826/1620] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.150223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.156273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 795/1621] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.159449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.160407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104777c-2a7a-4889-b6a7-b536ec2e4142 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.165191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.166041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.166187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.166545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.179149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.181075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.192929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190d5d-9594-4c34-a7f1-a53891ebf23b service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.195712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 827/1622] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.196708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.207133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.207832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.210181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.210303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.210426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90645e6b-5790-40e1-8cf2-e5d10bd55100 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.216372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.217189 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.217569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.217924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.218727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.220755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.235677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 828/1623] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:33.240956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.256416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a031ae2-1a9e-4703-bc85-ee7eaea0c85b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.264292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.265347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.265816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.266775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] PATH_INFO: `/v3/projects/4ed366412e154359aefb1d2f568f1438/users/924546a2ff4b42b7aa14a34f138578b0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.292275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed366412e154359aefb1d2f568f1438, user_id=924546a2ff4b42b7aa14a34f138578b0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.293871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.303065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.305775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.316250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 829/1624] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:33 2022] PUT /identity/v3/projects/4ed366412e154359aefb1d2f568f1438/users/924546a2ff4b42b7aa14a34f138578b0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:33.320088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.330060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 796/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.400525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7cdec-0411-4f95-bbca-41f94c61e267 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.408325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.409765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.409765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.410044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.411042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.413358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.433293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 830/1626] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:33.434029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.457790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792c3cd-69d0-4433-84c0-6632c1c5a012 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.466302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.487941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.489123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.490842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] PATH_INFO: `/v3/projects/4ed366412e154359aefb1d2f568f1438/users/924546a2ff4b42b7aa14a34f138578b0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.493705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.495356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed366412e154359aefb1d2f568f1438, user_id=924546a2ff4b42b7aa14a34f138578b0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.499827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa5264-f85b-4412-8de3-8bb569f5df4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.527947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 797/1627] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:33 2022] PUT /identity/v3/projects/4ed366412e154359aefb1d2f568f1438/users/924546a2ff4b42b7aa14a34f138578b0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:01:33.530222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3602760-daff-4560-a779-3a2c46ed9a82 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.530524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3602760-daff-4560-a779-3a2c46ed9a82 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.530806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3602760-daff-4560-a779-3a2c46ed9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.595427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:33.631276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a3602760-daff-4560-a779-3a2c46ed9a82 None None] MFA Rules not processed for user `924546a2ff4b42b7aa14a34f138578b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:33.633768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a3602760-daff-4560-a779-3a2c46ed9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:33.637696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef706-6137-4d24-9576-8caf29dae8ea admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.758762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 798/1628] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:33.765113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112b25-b9d7-4b89-b26e-27e35e26dff1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.765417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112b25-b9d7-4b89-b26e-27e35e26dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.765758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112b25-b9d7-4b89-b26e-27e35e26dff1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.767255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 799/1629] 10.222.0.61 () {54 vars in 912 bytes} [Thu May 19 22:01:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:33.778973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53112b25-b9d7-4b89-b26e-27e35e26dff1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:33.804714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:33.834511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.835184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.835474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.835751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.877648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53112b25-b9d7-4b89-b26e-27e35e26dff1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:33.881750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:33.882411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.882778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.883053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.895484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.897218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a4223-9583-4af9-a230-d5da945e102b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.912270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 800/1630] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.914152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e32af-4ba9-4a23-b5f8-d4ac40915834 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:33.914502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e32af-4ba9-4a23-b5f8-d4ac40915834 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:33.914787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e32af-4ba9-4a23-b5f8-d4ac40915834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:33.946959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd2e32af-4ba9-4a23-b5f8-d4ac40915834 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:33.948221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd2e32af-4ba9-4a23-b5f8-d4ac40915834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:33.950719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:33.952642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:33.963861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 831/1631] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:33.973821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 801/1632] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:01:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:33.989659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:34.002291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7262f-9a2d-45df-9fc5-668277918492 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:34.009453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:34.010289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:34.010787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:34.011616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:34.026313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:34.028336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:34.037034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacaad-a4c5-4375-abc8-545cc6610538 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:34.037329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacaad-a4c5-4375-abc8-545cc6610538 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:34.037592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacaad-a4c5-4375-abc8-545cc6610538 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:34.039473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 802/1633] 10.222.0.61 () {54 vars in 912 bytes} [Thu May 19 22:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:34.060321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 832/1634] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:34.064543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36337ab-bd99-4587-a826-9973096f2462 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:34.064981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36337ab-bd99-4587-a826-9973096f2462 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:34.065337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36337ab-bd99-4587-a826-9973096f2462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:34.102247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a36337ab-bd99-4587-a826-9973096f2462 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:34.103484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a36337ab-bd99-4587-a826-9973096f2462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:34.129127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 803/1635] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:01:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:34.308055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:34.317786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a3f8-baba-4328-9afb-bf86bd29ee61 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:34.327022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:34.327689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:34.327973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:34.332804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:34.357479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:34.358929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:34.404737 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 833/1636] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:35.083879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:35.084446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:35.086375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:35.180302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] MFA Rules not processed for user `383ec9ad9bac4dcfb6cf6fc68a0db3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:35.190995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:35.391538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 804/1637] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:35.394050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:35.678622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db3140-654b-4d4c-b15b-b0b4e319043d service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:35.679173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:35.698141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:35.703099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfae3-285b-4e6c-8c1b-33db0792988b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:35.709579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:35.711207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:35.711207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:35.712729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:35.715473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:35.716577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:35.717030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:35.717501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] PATH_INFO: `/v3/users/758eb207b7214f0eb4b14022a1ea46f2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:35.719740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] RBAC: Authorizing `identity:delete_user(user_id=758eb207b7214f0eb4b14022a1ea46f2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:35.722634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:35.723016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:35.741114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:35.801527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 834/1638] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:35 2022] POST /identity/v3/projects => generated 405 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:35.808587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758eb207b7214f0eb4b14022a1ea46f2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:35.808814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:35.829558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758eb207b7214f0eb4b14022a1ea46f2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:35.830430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758eb207b7214f0eb4b14022a1ea46f2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:35.831237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758eb207b7214f0eb4b14022a1ea46f2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:35.878501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758eb207b7214f0eb4b14022a1ea46f2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:35.910141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:35.922938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 805/1639] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:35 2022] DELETE /identity/v3/users/758eb207b7214f0eb4b14022a1ea46f2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:35.925316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:35.931697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.006414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df30ec3-9956-4bf2-a299-ecd8045f731f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.104210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ccd8-01a1-466d-8abd-17d66a3b797f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.111607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.112494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.112896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.113257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] PATH_INFO: `/v3/users/8e78bf605b25422da6f29b5dc95c0b6f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.114752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e78bf605b25422da6f29b5dc95c0b6f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:36.116851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:36.182294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.182502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e78bf605b25422da6f29b5dc95c0b6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:36.185584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.186054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.187411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.189031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:36.191177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af4f98-294d-48f6-8e3c-d978edcd3d22 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:36.191914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e78bf605b25422da6f29b5dc95c0b6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:36.192497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e78bf605b25422da6f29b5dc95c0b6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:36.193169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e78bf605b25422da6f29b5dc95c0b6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:36.214462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2471f8bd-e6d4-4cde-8402-733ac6f4cdbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e78bf605b25422da6f29b5dc95c0b6f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:36.235801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 806/1640] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:35 2022] DELETE /identity/v3/users/8e78bf605b25422da6f29b5dc95c0b6f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:36.237346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394546-3130-48d5-9e8e-7cd2fb5d479a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.237648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394546-3130-48d5-9e8e-7cd2fb5d479a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.237936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394546-3130-48d5-9e8e-7cd2fb5d479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.244599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 835/1641] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:35 2022] POST /identity/v3/users => generated 309 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:01:36.246524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b1ad5-203d-4f06-8813-e7c8ad4ab6a7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.247065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b1ad5-203d-4f06-8813-e7c8ad4ab6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.258723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b1ad5-203d-4f06-8813-e7c8ad4ab6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.269014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c8394546-3130-48d5-9e8e-7cd2fb5d479a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:36.270329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8394546-3130-48d5-9e8e-7cd2fb5d479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.364656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-004b1ad5-203d-4f06-8813-e7c8ad4ab6a7 None None] MFA Rules not processed for user `980d1f84ba3549e3b79ddbff8e090538`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:36.366555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-004b1ad5-203d-4f06-8813-e7c8ad4ab6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.463766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 807/1642] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:36.465932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.466241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.466518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.485390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:36.486949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.495341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 808/1643] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:36.499921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:36.501401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.531445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 836/1644] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:36.533481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.533957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.534632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.546767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982e17f-9b67-42b1-8b36-a185fb3ea129 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.560691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.561376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.561658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.562128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.562886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:36.564554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:36.570509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] MFA Rules not processed for user `980d1f84ba3549e3b79ddbff8e090538`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:36.572153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.598090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 809/1645] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:36.599194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:36.605703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 837/1646] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:01:36.606688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:36.611341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.616252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:36.623167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee65eeb-03b4-4d36-aecb-04a9176306ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.632250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:36.653125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.654049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.654327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.654602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] PATH_INFO: `/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f/users/2fe05ae28bc74d31b78a7fabcce52897/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.689536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a36fc43586e4dd48e2b0bf72b42a88f, user_id=2fe05ae28bc74d31b78a7fabcce52897, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:36.693574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:36.722152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e946e-9640-4222-bc11-5a5c8b4b879e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.724691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 810/1647] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:36 2022] PUT /identity/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f/users/2fe05ae28bc74d31b78a7fabcce52897/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:36.726283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:36.733402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:36.858509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.859198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.859491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.859769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:36.928100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a9d95-fae0-4d0d-950f-feccd7d6c794 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:36.946035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:36.946877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:36.947312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:36.947977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:37.047096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.058141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.096649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 838/1648] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:37.097596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:37.112053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.117704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.121701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e660f-c129-40f9-a402-fd2929313ef1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:37.132335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:37.134871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:37.135361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:37.135645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:37.141453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 811/1649] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:37.142505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:37.164226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5276-fb9f-43b0-bd77-3a6be28375d2 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:37.178719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:37.179500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:37.179791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:37.180144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:37.181000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.181683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.183059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.186984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.215725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 839/1650] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:37.221839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 812/1651] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:37.227029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:37.239650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4e72-5b89-4a56-962c-0de1fe1e5f90 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:37.257507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:37.258514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:37.258800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:37.259078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] PATH_INFO: `/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f/users/2fe05ae28bc74d31b78a7fabcce52897/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:37.263740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a36fc43586e4dd48e2b0bf72b42a88f, user_id=2fe05ae28bc74d31b78a7fabcce52897, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.266709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32955fa9-bede-41bc-a872-05d69d5c7eee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.300277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 840/1652] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:37 2022] PUT /identity/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f/users/2fe05ae28bc74d31b78a7fabcce52897/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:37.827765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:37.827765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:37.854845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c873a-edb3-4930-a34f-37603df3c1df admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:37.978450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:37.979357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:37.979642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:37.979926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] PATH_INFO: `/v3/users/214c6759db6c466cb132fefd2ddea57b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:37.981838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=214c6759db6c466cb132fefd2ddea57b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:37.983948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:37.989623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:38.151584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214c6759db6c466cb132fefd2ddea57b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:38.177229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214c6759db6c466cb132fefd2ddea57b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:38.178111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214c6759db6c466cb132fefd2ddea57b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:38.178842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214c6759db6c466cb132fefd2ddea57b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:38.217371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b019b20-ccda-4731-abe7-34a000ea8d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214c6759db6c466cb132fefd2ddea57b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:38.239241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 813/1653] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:37 2022] DELETE /identity/v3/users/214c6759db6c466cb132fefd2ddea57b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:38.244544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.245143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.245143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.295763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:38.297112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:38.313837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524723b-f85e-4df0-9fb8-c154bb47496b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.314075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524723b-f85e-4df0-9fb8-c154bb47496b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.314539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524723b-f85e-4df0-9fb8-c154bb47496b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.316023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 814/1654] 10.222.0.61 () {54 vars in 912 bytes} [Thu May 19 22:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 22:01:38.330325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.330691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.331053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.409894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:38.410881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:38.412253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:38.467295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 841/1655] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:38.497950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:38.499436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:38.547521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 815/1656] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:01:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:01:38.614088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49209a00-5fe5-47a1-b0dd-6197b2ecb516 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:38.620669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:38.621609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.621900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.622183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.640962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:38.642352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:38.657437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 842/1657] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:01:38.716541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:38.718927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:38.743680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479963e1-f198-474c-a772-5b6b4fdf242a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:38.752035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:38.752926 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.753353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.753641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.764887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:38.767044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f69dd7-a529-4576-93a9-13564c7b2abf service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:38.780999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 816/1658] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:38.875643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:38.878266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:38.929442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506bbcb-fede-4c2f-b177-433a1c8df14d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:38.941086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:38.942102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:38.942240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:38.942524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] PATH_INFO: `/v3/projects/26de7fe4dc7c417f9f7a3344e6d049d3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:38.946409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26de7fe4dc7c417f9f7a3344e6d049d3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:38.948718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.125206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26de7fe4dc7c417f9f7a3344e6d049d3', 'request_id': 'req-1a4079dd-ffde-486d-a7c0-25037610b628'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.139262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26de7fe4dc7c417f9f7a3344e6d049d3', 'request_id': 'req-1a4079dd-ffde-486d-a7c0-25037610b628'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.164079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26de7fe4dc7c417f9f7a3344e6d049d3', 'request_id': 'req-1a4079dd-ffde-486d-a7c0-25037610b628'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.187812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] The token cache is being invalidate because project 26de7fe4dc7c417f9f7a3344e6d049d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:39.188187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.190065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1a4079dd-ffde-486d-a7c0-25037610b628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.192412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 843/1659] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:38 2022] DELETE /identity/v3/projects/26de7fe4dc7c417f9f7a3344e6d049d3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:39.197711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.198161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.198322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.257714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:39.261251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.370020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 817/1660] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:39.377086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.377542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.377839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.409000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:39.410605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.423634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 844/1661] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:39.427530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:39.438336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25257b-1f1b-414f-82e3-c28ef1f55dea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:39.444217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:39.444955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.445245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.445528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.446288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:39.446602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:39.448267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.452725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:39.490615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 818/1662] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:39.495605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:39.516426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f951ed-c04d-4ed4-95d7-c2d633ca5d8f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:39.534894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:39.535881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.536245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.536634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.537462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:39.539599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.581967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:39.588177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.621145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2894b6-daec-4315-acc3-4d0656891ad5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:39.627523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 845/1663] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:39 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:39.632985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:39.636495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.744227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:39.747844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.748403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.749414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] PATH_INFO: `/v3/projects/bb177f0cfe3a432c80d389c2e048e82e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.770033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb177f0cfe3a432c80d389c2e048e82e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:39.770165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:39.773373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:39.773373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) May 19 22:01:39.773373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: May 19 22:01:39.779587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.816103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4daf-4e7e-49f4-82bb-a0514d50cf10 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:39.830784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:39.832874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.834275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.834557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] PATH_INFO: `/v3/projects/12d00f7cc000446d8c2f294215d691e5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.836804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d00f7cc000446d8c2f294215d691e5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:39.844338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.936378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb177f0cfe3a432c80d389c2e048e82e', 'request_id': 'req-ee0e4cee-d68c-4801-9038-b014b718aae5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:39.998638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb177f0cfe3a432c80d389c2e048e82e', 'request_id': 'req-ee0e4cee-d68c-4801-9038-b014b718aae5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.009559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d00f7cc000446d8c2f294215d691e5', 'request_id': 'req-0fc3892d-c849-42c6-811e-5273f7248b33'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.018670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb177f0cfe3a432c80d389c2e048e82e', 'request_id': 'req-ee0e4cee-d68c-4801-9038-b014b718aae5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.036092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d00f7cc000446d8c2f294215d691e5', 'request_id': 'req-0fc3892d-c849-42c6-811e-5273f7248b33'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.056865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d00f7cc000446d8c2f294215d691e5', 'request_id': 'req-0fc3892d-c849-42c6-811e-5273f7248b33'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.064575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] The token cache is being invalidate because project bb177f0cfe3a432c80d389c2e048e82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:40.066862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.069164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.078590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 819/1664] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:39 2022] DELETE /identity/v3/projects/bb177f0cfe3a432c80d389c2e048e82e => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:40.082467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:40.088268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.102428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] The token cache is being invalidate because project 12d00f7cc000446d8c2f294215d691e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:40.102740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.108987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0fc3892d-c849-42c6-811e-5273f7248b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.114669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 846/1665] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:39 2022] DELETE /identity/v3/projects/12d00f7cc000446d8c2f294215d691e5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:40.125464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.125762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.126058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.197265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:40.202707 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.330644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4cee-d68c-4801-9038-b014b718aae5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:40.485624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 847/1666] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:40.486855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:40.488930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:40.494611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.549585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e4a0b-5fff-4719-a7a7-072810d5aa87 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:40.560177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:40.561438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.561772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.562059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] PATH_INFO: `/v3/users/924546a2ff4b42b7aa14a34f138578b0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.563648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=924546a2ff4b42b7aa14a34f138578b0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:40.565784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 22:01:40.565784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) May 19 22:01:40.565784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: May 19 22:01:40.569056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:40.587164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:40.588207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.588207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.588463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.589387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:40.595752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d828-f205-4174-a0ba-11d29027d455 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:40.646611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 820/1667] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:40 2022] POST /identity/v3/users => generated 303 bytes in 567 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:40.649614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d5a99-bc5c-4c78-b617-3f79190377f1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.649917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d5a99-bc5c-4c78-b617-3f79190377f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.650207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d5a99-bc5c-4c78-b617-3f79190377f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.697278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad3d5a99-bc5c-4c78-b617-3f79190377f1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:40.704464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad3d5a99-bc5c-4c78-b617-3f79190377f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.724752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 821/1668] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:40.738063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.738391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924546a2ff4b42b7aa14a34f138578b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.738479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.738923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.742542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 822/1669] 10.222.0.61 () {54 vars in 960 bytes} [Thu May 19 22:01:40 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:40.745528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:40.752934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.763706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924546a2ff4b42b7aa14a34f138578b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.764295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924546a2ff4b42b7aa14a34f138578b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.795439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131430c-a385-4151-ba98-91c4056e9a5c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:40.803936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:40.804647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.804951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.805428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.805863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924546a2ff4b42b7aa14a34f138578b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.806236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:40.808440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:40.829750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 823/1670] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:40.831033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:40.831921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:40.876663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384889-489f-452c-a432-672ac07e6e88 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:40.879054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-145015c3-7f27-454c-a548-df04d00a9cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924546a2ff4b42b7aa14a34f138578b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:40.893461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 848/1671] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:40 2022] DELETE /identity/v3/users/924546a2ff4b42b7aa14a34f138578b0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:40.894538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:40.895323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.895713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.896708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.897567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:40.898634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:40.899062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:40.900827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:40.901223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:40.901617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:40.935715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 824/1672] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:40.935715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:40.943934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:40.953399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.957377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:40.982389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1e7c3-7286-450d-bd42-3d0ce2f42570 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.133644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:41.134346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.134637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.135289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.135767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:41.139305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:41.149366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 849/1673] 10.222.0.61 () {58 vars in 1057 bytes} [Thu May 19 22:01:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:41.150696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:41.153830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:41.173319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 825/1674] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:41.173319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:41.191439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaff559-2096-4be0-82dd-0934d4e8b13d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.192311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d815-2bdd-4698-8e12-9a4e9c7d0e8c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.205918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:41.206588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.206877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.207249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.208206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:41.209567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:41.211852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.212330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.215986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.217021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:41.219436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:41.233141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:41.235773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:41.248933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 826/1675] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:41.250269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:41.285513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46e78-e4fb-460c-9123-7c74bb129dca service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.292192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 850/1676] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:41 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:41.297997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:41.422010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647dc6-c958-49cc-b8d4-57de5f245032 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.437115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:41.438352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.438795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.439214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.440180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:41.443810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:41.486515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 851/1677] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:41 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:41.490209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:41.490933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:41.491885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.492169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.492447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] PATH_INFO: `/v3/projects/a76d5a0b0150413aae78f003f087c12e/users/e1e0263530f54ba8b417027680f4d15c/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.520432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76d5a0b0150413aae78f003f087c12e, user_id=e1e0263530f54ba8b417027680f4d15c, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:41.524519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8734150-8eb2-460b-aa7b-9e024aef803c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:41.530454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:41.547615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1658b-4d0f-4c9c-a7e7-88b09c85fe37 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:41.562931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 827/1678] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:41 2022] PUT /identity/v3/projects/a76d5a0b0150413aae78f003f087c12e/users/e1e0263530f54ba8b417027680f4d15c/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:41.575802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359b34c-591a-4816-aefd-0bb4f162e7aa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.577082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359b34c-591a-4816-aefd-0bb4f162e7aa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.577612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359b34c-591a-4816-aefd-0bb4f162e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.659097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3359b34c-591a-4816-aefd-0bb4f162e7aa None None] MFA Rules not processed for user `e1e0263530f54ba8b417027680f4d15c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:41.660956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3359b34c-591a-4816-aefd-0bb4f162e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:41.767834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 828/1679] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:41.769638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:41.769783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:41.770066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:41.840092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] MFA Rules not processed for user `2fe05ae28bc74d31b78a7fabcce52897`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:41.841896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:42.104645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 829/1680] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:42.106660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.144922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:42.201704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74882628-6c80-48c1-9f6b-071c0fcd9017 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.208095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.208213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.208715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.208822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.209088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.209304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.209419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.209826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.210091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.210836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.213236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.221114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.247097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:42.279089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 852/1681] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 789 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:01:42.283722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.300134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 830/1682] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:42 2022] POST /identity/v3/projects => generated 405 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:42.304396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.321422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40598f1-0f8e-4cd6-bc62-568008dc4fd9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.493827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5efee-b4e6-4548-8fa6-b8c56504e7fa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.507862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.512042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.512513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.512660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.512978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.513359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.513659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.513950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.514997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.518023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.521013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.523189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.554062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 853/1683] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:42.555358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.578805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 831/1684] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:42 2022] POST /identity/v3/users => generated 309 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:01:42.582707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.588255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317f062-9768-4878-b967-70cfadd9e334 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.597539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.599409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.601100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.601442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] PATH_INFO: `/v3/projects/3f83ed8823724784b406f3b5acfcb32f/users/41d396bd1430439aabfcf5a6d887451f/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.607183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611eb18-08c6-4aa4-8600-3fde2359eab4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.619585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.620399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.620572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.621403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.622155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.627528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.656005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 832/1685] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:42.660462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.666859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f83ed8823724784b406f3b5acfcb32f, user_id=41d396bd1430439aabfcf5a6d887451f, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.679654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cbe13-c358-4475-8d2f-8164d3db11ae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.680408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8370200-4aa6-40d3-be55-5ef8c61cda3f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.697195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.698129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.698403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.699394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] PATH_INFO: `/v3/projects/d129aa5c06914c8097f1760c91c7bd44/users/9d3061e459e3438fa50fe90629479602/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.732951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 854/1686] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:42 2022] PUT /identity/v3/projects/3f83ed8823724784b406f3b5acfcb32f/users/41d396bd1430439aabfcf5a6d887451f/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:01:42.745061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.745356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.746483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.752296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d129aa5c06914c8097f1760c91c7bd44, user_id=9d3061e459e3438fa50fe90629479602, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.755352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.791403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 833/1687] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:42 2022] PUT /identity/v3/projects/d129aa5c06914c8097f1760c91c7bd44/users/9d3061e459e3438fa50fe90629479602/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:42.795465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:42.837957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] MFA Rules not processed for user `41d396bd1430439aabfcf5a6d887451f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:42.839291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:42.959278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7e8f9-ad7f-4395-a352-89b10b9181c2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:42.973394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:42.977049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:42.977496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:42.977790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:42.978563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:42.982020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:42.990477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 855/1688] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:42.991378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:43.004187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.034577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 834/1689] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:43.035626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:43.037609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.060908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaaf7c-5b19-491f-9707-52eddd12ac67 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:43.087534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:43.087534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.087534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.087534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] PATH_INFO: `/v3/projects/4ed366412e154359aefb1d2f568f1438` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.088017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed366412e154359aefb1d2f568f1438)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:43.091591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:43.120499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:43.130206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77787ec7-65d9-4e0c-89d6-81665ffc7150 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:43.135805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:43.136513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.137618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.137618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.150504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:43.152269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c5ae-a948-4c4d-a3ef-e82a8d3314fe service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:43.163590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 835/1690] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:43.164961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041db8-e380-441f-956e-f8c58ee6658d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.165256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041db8-e380-441f-956e-f8c58ee6658d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.165537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041db8-e380-441f-956e-f8c58ee6658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.197841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f2041db8-e380-441f-956e-f8c58ee6658d None None] MFA Rules not processed for user `7ef029bafd694b46973cdff0c87123e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:43.200386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2041db8-e380-441f-956e-f8c58ee6658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.225258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:43.248883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed366412e154359aefb1d2f568f1438', 'request_id': 'req-ddcc68ec-e735-4914-be45-b8a7d912fb84'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:43.263721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed366412e154359aefb1d2f568f1438', 'request_id': 'req-ddcc68ec-e735-4914-be45-b8a7d912fb84'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:43.276163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed366412e154359aefb1d2f568f1438', 'request_id': 'req-ddcc68ec-e735-4914-be45-b8a7d912fb84'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:43.297176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] The token cache is being invalidate because project 4ed366412e154359aefb1d2f568f1438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:43.297735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:43.298658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:43.300485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 856/1691] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:42 2022] DELETE /identity/v3/projects/4ed366412e154359aefb1d2f568f1438 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:43.301465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:43.303129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.336684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 836/1692] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:43.336684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd27a-926d-4892-af1d-0f073973b406 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.336684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd27a-926d-4892-af1d-0f073973b406 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.336684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd27a-926d-4892-af1d-0f073973b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.380096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cdebd27a-926d-4892-af1d-0f073973b406 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:43.381928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cdebd27a-926d-4892-af1d-0f073973b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.474954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc68ec-e735-4914-be45-b8a7d912fb84 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:43.487222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:43.490772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.491083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.491365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] PATH_INFO: `/v3/projects/d129aa5c06914c8097f1760c91c7bd44/users/9d3061e459e3438fa50fe90629479602/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.496462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d129aa5c06914c8097f1760c91c7bd44, user_id=9d3061e459e3438fa50fe90629479602, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:43.505911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62d930-722c-4598-ba84-3046bbb52d48 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:43.529701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 837/1693] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:43.540321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a556-9ee3-4b16-b8d2-36d28dbc956d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.540551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a556-9ee3-4b16-b8d2-36d28dbc956d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.541456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a556-9ee3-4b16-b8d2-36d28dbc956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.542328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 857/1694] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:43 2022] PUT /identity/v3/projects/d129aa5c06914c8097f1760c91c7bd44/users/9d3061e459e3438fa50fe90629479602/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:43.544071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.544388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.544694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.585618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8f17a556-9ee3-4b16-b8d2-36d28dbc956d None None] MFA Rules not processed for user `7ef029bafd694b46973cdff0c87123e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:43.590300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f17a556-9ee3-4b16-b8d2-36d28dbc956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.594531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:43.595866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.720361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 838/1695] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:43.721454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.721702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.721984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:43.747432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 858/1696] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:43.754891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:43.759530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.825880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] MFA Rules not processed for user `9d3061e459e3438fa50fe90629479602`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:43.831148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:43.868532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:43.968876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:43.985000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c616f7c-5367-4b15-b6a0-892efd2557a9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:43.995896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:43.996571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:43.996885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:43.997163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.023125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.024862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.049490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 859/1697] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:44.050505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.077983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 839/1698] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:44.079034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.080661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.086629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149adbc-f7d0-48fd-9aec-4f104108c5f1 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.094711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.094711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.094989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.095248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.095876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:44.096181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.097808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.100064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa4fa-b3ce-4ece-bd25-64a523892a0a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.113396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.114625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.121119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.121119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] PATH_INFO: `/v3/users/6ae4e8e69a424bed9a271d567a8b65ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.126646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae4e8e69a424bed9a271d567a8b65ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.130056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.151243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 860/1699] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:44.154794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.162390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.290220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae4e8e69a424bed9a271d567a8b65ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:44.312900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae4e8e69a424bed9a271d567a8b65ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:44.314520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae4e8e69a424bed9a271d567a8b65ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:44.315071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae4e8e69a424bed9a271d567a8b65ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:44.335084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eaeb4-8bb6-405a-a1e5-b75db79d3599 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.340386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.342548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.343301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.343301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.346699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.366991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.368922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.370204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51b47283-3b7b-4b9f-a537-6cb31a08c6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae4e8e69a424bed9a271d567a8b65ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:44.397395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 861/1701] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:44 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:44.399978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 840/1701] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:44 2022] DELETE /identity/v3/users/6ae4e8e69a424bed9a271d567a8b65ec => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:44.401042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.401868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.403055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.403577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.406638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.427981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:44.429708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.460207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f982e-d098-4d8d-9fc4-7954d55f8251 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.573024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 841/1702] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:44.618666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.620460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.621543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.621954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.622921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.625660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.627345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.634586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.688365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:44.706585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 862/1703] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:44 2022] POST /identity/v3/projects => generated 363 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:44.711537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:44.717118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:44.813705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f867be-3a31-4ce4-9deb-f952bd616bb6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.873920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7c4b9-e086-422a-82c0-a598e0305bf9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:44.893591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.894674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.895072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.896981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] PATH_INFO: `/v3/users/e1e0263530f54ba8b417027680f4d15c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:44.900407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e0263530f54ba8b417027680f4d15c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:44.906163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:44.926002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:44.969880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:44.974738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:44.975046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:44.976781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.018031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:45.019729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:45.056017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e0263530f54ba8b417027680f4d15c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:45.067406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e0263530f54ba8b417027680f4d15c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:45.068151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e0263530f54ba8b417027680f4d15c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:45.077526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 842/1704] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:45.078727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.086638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e0263530f54ba8b417027680f4d15c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:45.099310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae19-7e67-4857-adc2-45716d576ae1 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:45.111108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:45.111999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.112192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.112472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.113205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:45.114120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-af055ee4-2498-450c-b06c-78323981f044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e0263530f54ba8b417027680f4d15c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:45.115184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:45.126274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 863/1705] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:44 2022] DELETE /identity/v3/users/e1e0263530f54ba8b417027680f4d15c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:45.127603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec436cf8-8cec-42c0-aee1-715c77a96bf0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.127961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec436cf8-8cec-42c0-aee1-715c77a96bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.128253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec436cf8-8cec-42c0-aee1-715c77a96bf0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.145194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 864/1706] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:45.148142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622d5aa-9943-410a-a5cb-bbec1053818e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.150211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622d5aa-9943-410a-a5cb-bbec1053818e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.152679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622d5aa-9943-410a-a5cb-bbec1053818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.163830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 843/1707] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:45 2022] POST /identity/v3/users => generated 282 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:45.164868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.170765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.194630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bf952-9f5a-4423-84fc-f38d32b9d8e0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:45.203668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d622d5aa-9943-410a-a5cb-bbec1053818e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:45.205392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d622d5aa-9943-410a-a5cb-bbec1053818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.341221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:45.342151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.342266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.348315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.350384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:45.358836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:45.393296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 865/1708] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:45.393989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.395625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.395860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 844/1709] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:45.395955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.397151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.427246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46509821-6959-41f0-b920-295cec4b6870 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:45.440750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:45.441549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.442037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.442549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.443788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:45.446955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275744d-6c6a-4a89-8995-291ce90cabf8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:45.455578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:45.457058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.484162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 845/1710] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:45.487441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.487794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.488098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.538319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:45.540526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.663459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 866/1711] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:01:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:45.664150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.667823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.673434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:45.702574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 846/1712] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:01:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:45.703359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.737443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc2749-8a70-4a38-9d2d-4b82fd087000 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:45.741223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:45.761667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:45.763582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:45.764423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:45.766013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] PATH_INFO: `/v3/projects/4236d30e836f494792c28958d5823bb5/users/1ac6f3fd1a3a4a1495b0b329db9c2fed/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:45.813404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4236d30e836f494792c28958d5823bb5, user_id=1ac6f3fd1a3a4a1495b0b329db9c2fed, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:45.816720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:45.842132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 847/1713] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:45 2022] PUT /identity/v3/projects/4236d30e836f494792c28958d5823bb5/users/1ac6f3fd1a3a4a1495b0b329db9c2fed/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:45.843200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:45.859641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:45.881813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e669c-f7ed-4c00-92b8-f261cbca0bda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.032374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebdf29-45c7-4033-bd33-cf885e76bf8a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.046714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.047745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.048148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.048882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.054554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.055844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.056154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.056440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.073077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.079823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7759f63-1bee-4019-8472-1a20b96309e2 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.113550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 848/1714] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:46.115359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3843de-ac07-46f7-8d8b-370e4ba6c0b9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.115879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3843de-ac07-46f7-8d8b-370e4ba6c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.116271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3843de-ac07-46f7-8d8b-370e4ba6c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.194453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ce3843de-ac07-46f7-8d8b-370e4ba6c0b9 None None] MFA Rules not processed for user `1ac6f3fd1a3a4a1495b0b329db9c2fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:46.195813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce3843de-ac07-46f7-8d8b-370e4ba6c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:46.264551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.266057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be679277-5e7d-4141-9348-46d4ec83ffdb service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.300349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 867/1715] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:46.301610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.301946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.302260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.303715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 868/1716] 10.222.0.61 () {54 vars in 912 bytes} [Thu May 19 22:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:46.304407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:46.309933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:46.433908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 849/1717] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:46.435449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.435741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.436020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.480493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:46.482178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:46.492856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 850/1718] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:01:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:46.493980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:46.514739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7bde7-0378-4fb5-a4e6-4ae63d128440 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.521403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.522146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.522489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.522770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.523956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.526478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.552880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c39465-03d7-46a6-9b24-c73a8547be22 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.573342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.574248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.574687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.574972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.640284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 851/1719] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:46 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:46.643259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:46.660030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:46.695354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:46.720732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.723305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.759141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362500cf-d350-4e96-a2f5-d7c58d026dc2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.783594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.784669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.785119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.785334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.786243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.790135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.802469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 869/1720] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:01:46.827917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:46.829749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:46.876045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 852/1721] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:46 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:46.887896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450e32-aa9c-4299-b5ad-7f8f43ec27e6 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.888216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:46.895240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.896142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.896424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.896774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] PATH_INFO: `/v3/users/383ec9ad9bac4dcfb6cf6fc68a0db3b0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.898012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=383ec9ad9bac4dcfb6cf6fc68a0db3b0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.903390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:46.940646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f186f64-43bc-491a-bd9f-bbd50179ec62 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:46.964195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:46.965064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:46.965357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:46.965656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:46.966998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:46.968969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:47.017316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383ec9ad9bac4dcfb6cf6fc68a0db3b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:47.034400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383ec9ad9bac4dcfb6cf6fc68a0db3b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:47.036410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383ec9ad9bac4dcfb6cf6fc68a0db3b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:47.051104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 853/1722] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:47.055001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:47.061517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383ec9ad9bac4dcfb6cf6fc68a0db3b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:47.087949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc205-5c01-4720-b8b8-901a76ee44bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:47.110895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:47.117671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:47.117966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:47.118249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] PATH_INFO: `/v3/projects/33103e09bc8b47519c10937369742e0a/users/2b3806ceb1d84c5e8fe606e442e88618/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:47.121978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383ec9ad9bac4dcfb6cf6fc68a0db3b0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:47.159632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 870/1723] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:46 2022] DELETE /identity/v3/users/383ec9ad9bac4dcfb6cf6fc68a0db3b0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:47.159632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:47.168205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:47.179957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=33103e09bc8b47519c10937369742e0a, user_id=2b3806ceb1d84c5e8fe606e442e88618, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:47.189340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:47.244432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 854/1724] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:47 2022] PUT /identity/v3/projects/33103e09bc8b47519c10937369742e0a/users/2b3806ceb1d84c5e8fe606e442e88618/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:47.247934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:47.254130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:47.428300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510fd56-71ec-4dec-ab75-f35be1066d44 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:47.551300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a5743-1c5d-407a-803f-bb4ba1cff69a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:47.565168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:47.565873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:47.566158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:47.566437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:47.578207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:47.582953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:47.609201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:47.609801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:47.610089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:47.613425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:47.620262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 855/1725] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:47.621840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:47.622897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:47.650036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb491df4-bf7e-406b-a774-679dca811268 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:47.662798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:47.664314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2910c-24fb-4e18-859f-e92b7ef2e9f2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:47.680870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:47.681431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 871/1726] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:47.699383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:47.700433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:47.700896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:47.701216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] PATH_INFO: `/v3/projects/33103e09bc8b47519c10937369742e0a/users/2b3806ceb1d84c5e8fe606e442e88618/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:47.703727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33103e09bc8b47519c10937369742e0a, user_id=2b3806ceb1d84c5e8fe606e442e88618, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:47.707412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:47.744878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 856/1727] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:47 2022] PUT /identity/v3/projects/33103e09bc8b47519c10937369742e0a/users/2b3806ceb1d84c5e8fe606e442e88618/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:47.758071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:47.758712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:47.760460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:47.867139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] MFA Rules not processed for user `2b3806ceb1d84c5e8fe606e442e88618`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:47.868720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:47.933590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:47.936267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:48.211457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:48.228480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffb283-e3d5-4f69-b021-de196fb7222b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:48.277500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 872/1728] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:47 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:48.279586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:48.281910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:48.302300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:48.305676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:48.309488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:48.314512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] PATH_INFO: `/v3/projects/f9b4725e31b24d4fbdcbf849cfbb7f45` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:48.316352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b4725e31b24d4fbdcbf849cfbb7f45)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:48.327429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:48.365416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd4f2-7546-4544-a04e-568018df7bb0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:48.388855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:48.390515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:48.391830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:48.392506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] PATH_INFO: `/v3/projects/a76d5a0b0150413aae78f003f087c12e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:48.401182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a76d5a0b0150413aae78f003f087c12e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:48.411582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:48.473169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b4725e31b24d4fbdcbf849cfbb7f45', 'request_id': 'req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.498364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b4725e31b24d4fbdcbf849cfbb7f45', 'request_id': 'req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.560872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b4725e31b24d4fbdcbf849cfbb7f45', 'request_id': 'req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.588475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] The token cache is being invalidate because project f9b4725e31b24d4fbdcbf849cfbb7f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:48.590986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.593575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.600596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 857/1729] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:47 2022] DELETE /identity/v3/projects/f9b4725e31b24d4fbdcbf849cfbb7f45 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:48.601263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:48.613516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:48.700399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a76d5a0b0150413aae78f003f087c12e', 'request_id': 'req-e1ceff32-f18a-42bc-aa11-7240d8a319cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.736365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e56fb8-8ebb-4479-9ff0-c5ed283893ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:48.739859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a76d5a0b0150413aae78f003f087c12e', 'request_id': 'req-e1ceff32-f18a-42bc-aa11-7240d8a319cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.776304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a76d5a0b0150413aae78f003f087c12e', 'request_id': 'req-e1ceff32-f18a-42bc-aa11-7240d8a319cc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.825481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] The token cache is being invalidate because project a76d5a0b0150413aae78f003f087c12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:48.825777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.829239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:48.830346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:48.830721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:48.830925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:48.833887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:48.834239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:48.836978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 873/1730] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:48 2022] DELETE /identity/v3/projects/a76d5a0b0150413aae78f003f087c12e => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:48.847778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:48.854797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:49.005572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:49.015007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:49.016959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:49.043380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 858/1731] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:49.044718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:49.046872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:49.061930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceff32-f18a-42bc-aa11-7240d8a319cc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:49.076778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:49.080530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:49.080886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:49.082549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] PATH_INFO: `/v3/users/2fe05ae28bc74d31b78a7fabcce52897` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:49.087183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe05ae28bc74d31b78a7fabcce52897)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:49.092367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9975fd-5494-41e3-b67b-f4c3c583e407 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:49.092866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:49.107213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:49.108985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:49.110168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:49.110533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:49.111313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] PATH_INFO: `/v3/users/9d3061e459e3438fa50fe90629479602` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:49.113578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3061e459e3438fa50fe90629479602)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:49.117725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:49.239389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe05ae28bc74d31b78a7fabcce52897'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.255998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3061e459e3438fa50fe90629479602'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.257692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe05ae28bc74d31b78a7fabcce52897'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.258477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe05ae28bc74d31b78a7fabcce52897'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.276447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe05ae28bc74d31b78a7fabcce52897'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.290003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3061e459e3438fa50fe90629479602'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.291006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3061e459e3438fa50fe90629479602'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.291679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3061e459e3438fa50fe90629479602'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.308454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe05ae28bc74d31b78a7fabcce52897'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.315447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 874/1732] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:48 2022] DELETE /identity/v3/users/2fe05ae28bc74d31b78a7fabcce52897 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:49.318490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:49.332406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:49.346070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3061e459e3438fa50fe90629479602'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:49.409064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 859/1733] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:49 2022] DELETE /identity/v3/users/9d3061e459e3438fa50fe90629479602 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:49.606893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d556-1e37-42e6-9674-276fe391fdca admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:49.704297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:49.705088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:49.705502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:49.705719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:49.708115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:49.884223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:49.892476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:49.894888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb1e62-249c-4f17-a498-9890069849d0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:49.921470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 875/1734] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:50.330536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:50.335621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:50.702208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63032a-78e5-4312-9499-09f62435293f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:50.716784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:50.717684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:50.717955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:50.718231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] PATH_INFO: `/v3/projects/4a9d5bf488674c00a7c1ed3e678ce29c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:50.721041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9d5bf488674c00a7c1ed3e678ce29c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:50.723030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:50.816041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9d5bf488674c00a7c1ed3e678ce29c', 'request_id': 'req-3f8399f2-4dfc-487f-8872-4bcbc2390392'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:50.824515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9d5bf488674c00a7c1ed3e678ce29c', 'request_id': 'req-3f8399f2-4dfc-487f-8872-4bcbc2390392'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:50.843350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9d5bf488674c00a7c1ed3e678ce29c', 'request_id': 'req-3f8399f2-4dfc-487f-8872-4bcbc2390392'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:50.854969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:50.874764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] The token cache is being invalidate because project 4a9d5bf488674c00a7c1ed3e678ce29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:50.875160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:50.877553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:50.885875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 860/1735] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:50 2022] DELETE /identity/v3/projects/4a9d5bf488674c00a7c1ed3e678ce29c => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:50.888678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:50.889543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:50.890063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:50.917020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:50.918987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:50.997303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:51.000400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:51.157567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 876/1736] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:51.165782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:51.172078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:51.242899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e73e45-560f-400c-8717-f6cb10cb32cb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:51.259330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:51.263574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.264249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.264770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] PATH_INFO: `/v3/projects/d129aa5c06914c8097f1760c91c7bd44` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.267083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d129aa5c06914c8097f1760c91c7bd44)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:51.274445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:51.324264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8399f2-4dfc-487f-8872-4bcbc2390392 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:51.329594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:51.330491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.330946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.331568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] PATH_INFO: `/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.333180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a36fc43586e4dd48e2b0bf72b42a88f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:51.335600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:51.424954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a36fc43586e4dd48e2b0bf72b42a88f', 'request_id': 'req-a2d02411-aa41-48cd-9f0f-af597e8414a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.430095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd129aa5c06914c8097f1760c91c7bd44', 'request_id': 'req-01b8fb3f-aa69-4021-b166-f21b17b94ef8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.443511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a36fc43586e4dd48e2b0bf72b42a88f', 'request_id': 'req-a2d02411-aa41-48cd-9f0f-af597e8414a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.445783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd129aa5c06914c8097f1760c91c7bd44', 'request_id': 'req-01b8fb3f-aa69-4021-b166-f21b17b94ef8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.460846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a36fc43586e4dd48e2b0bf72b42a88f', 'request_id': 'req-a2d02411-aa41-48cd-9f0f-af597e8414a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.467790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd129aa5c06914c8097f1760c91c7bd44', 'request_id': 'req-01b8fb3f-aa69-4021-b166-f21b17b94ef8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.546503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] The token cache is being invalidate because project 1a36fc43586e4dd48e2b0bf72b42a88f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:51.548541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.551219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a2d02411-aa41-48cd-9f0f-af597e8414a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.552071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] The token cache is being invalidate because project d129aa5c06914c8097f1760c91c7bd44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:01:51.553617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.554582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 861/1737] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:50 2022] DELETE /identity/v3/projects/1a36fc43586e4dd48e2b0bf72b42a88f => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:51.555019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-01b8fb3f-aa69-4021-b166-f21b17b94ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:51.559942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddd06c-83ea-4c12-8a15-f899453045d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.561558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddd06c-83ea-4c12-8a15-f899453045d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.568019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 877/1738] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:01:51 2022] DELETE /identity/v3/projects/d129aa5c06914c8097f1760c91c7bd44 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:51.568659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddd06c-83ea-4c12-8a15-f899453045d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.575052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb86d-0ce6-428b-a036-726231663f7a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.575909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb86d-0ce6-428b-a036-726231663f7a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.576669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb86d-0ce6-428b-a036-726231663f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.631940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-51ddd06c-83ea-4c12-8a15-f899453045d8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:51.635585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51ddd06c-83ea-4c12-8a15-f899453045d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:51.638121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b6acb86d-0ce6-428b-a036-726231663f7a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:51.641689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6acb86d-0ce6-428b-a036-726231663f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:51.806945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 862/1739] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:51.811338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.813521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.815419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.860142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:51.861538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:51.889246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 863/1740] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:51.898940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:51.906799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 878/1741] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:51.916298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.920656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.921163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668e8e-6d1a-4c50-ae82-fe2d64a36374 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:51.934466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:51.934861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.937989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:51.938301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:51.947093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:51.947678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:51.948024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:51.951489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94420b22-da5b-4d34-b614-395ca53b09ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:51.994961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:52.002342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 864/1742] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:52.002342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.002606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:52.003499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.004713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.040835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 879/1743] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:52.041755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.049426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:52.059160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:52.063156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:52.071020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec5a93-f0fa-41bd-a401-67cd24e72505 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.090916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.091791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.092077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.092355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.093091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.095192 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.148599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 865/1744] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:52.153281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.155878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:52.193159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66bdb-16ab-4360-87c1-e94f6651163f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.219118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.219956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.220379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.220507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.221381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:52.221682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.224147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.233066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 880/1745] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:52.233950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.274677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 866/1746] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:52.274677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.294589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbf97c-b813-4a46-bea4-96f360a27a76 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.388967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.389834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.389975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.390265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.391018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.394118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.434395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 867/1747] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:52.435201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.448694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67c71-67c6-4ced-a4a8-17e8e0746dc0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.456195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.457492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.457782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.458713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.460685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.463879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.503096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca566a-f698-4ac8-a0ee-301da090d6b4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.508778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.509594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.510004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.510437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.512012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:01:52.512012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.524384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.545584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 868/1748] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:52.546040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.550854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 881/1749] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:01:52.551777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.574675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17481ed-9fd0-4b04-b366-e04ae5bfb6f0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.714009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.715058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.715505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.715797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.716734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.719712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.745129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f612d4-c8d3-47e1-ac73-e96d3d4cbbd2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.753462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.754338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.754679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.755036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.755961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.758011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f158fe6-7e65-428d-8f5b-b891e06509d9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.763333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 882/1750] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:52.764888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.780763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 869/1751] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:52.785313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.785615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.785826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3bb8-a375-4265-9151-d2876354ec15 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:52.785959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.791531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:52.793110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.793459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.794666 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.795627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:52.797695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc786bc-dd13-4b2b-9245-270b1bfaa1da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:52.820735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] MFA Rules not processed for user `7ef029bafd694b46973cdff0c87123e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:52.820735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:52.893374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 883/1752] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:52.895086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8629-662a-4337-9374-712754e06b10 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:52.896437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8629-662a-4337-9374-712754e06b10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:52.897271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8629-662a-4337-9374-712754e06b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:52.934286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:52.967304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 870/1753] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:52.969825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:52.989093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-95ab8629-662a-4337-9374-712754e06b10 None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:52.993961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-95ab8629-662a-4337-9374-712754e06b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:52.996780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293711f-45f3-4447-a7a9-f3a5a1063de9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.221568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.222587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.231662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.232291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.235413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.237213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.265764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 871/1754] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:53.265764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.274382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 884/1755] 10.222.0.49 () {58 vars in 1015 bytes} [Thu May 19 22:01:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:53.275238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab8629-662a-4337-9374-712754e06b10 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.290949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c488-db10-4c2e-8860-026b08cac97d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.293429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:53.306770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.308727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.309266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab8629-662a-4337-9374-712754e06b10 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.309751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.311085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.311962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.317539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.328898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.329607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.329893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.330177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.330938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.334340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.346596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 872/1756] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:53.347509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.370855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fae88-f117-4c16-bfa2-f8a08b49791c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.379765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.381387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.381679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.381964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.383073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.385205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 885/1757] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:53 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:53.387020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.387359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.406639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 873/1758] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:53.406639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.418727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4e1c4-7d7b-4c63-95bb-0c4bb65ba118 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.424297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.424467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d8cf0-e03a-48c7-8ee4-419103f3bb86 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.425212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.425644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.425992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.426811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.428535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.444746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 874/1759] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:53.444746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.447933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.449539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.449817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.450104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] PATH_INFO: `/v3/projects/96a2d3d39f2a4920931044f8ee23d87b/users/78295aa21c2c4517aff1bbbb6f931f4e/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.457847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d845688-d9bb-4045-ae98-a093040a178d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.462877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.463902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.464777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.465128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] PATH_INFO: `/v3/projects/1fbb080b357b4ed6843106de52d88333/users/3a678e305861422389a75106c379fc4c/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.484172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbb080b357b4ed6843106de52d88333, user_id=3a678e305861422389a75106c379fc4c, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.487321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.489974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a2d3d39f2a4920931044f8ee23d87b, user_id=78295aa21c2c4517aff1bbbb6f931f4e, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.492764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.523097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 875/1760] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:53 2022] PUT /identity/v3/projects/1fbb080b357b4ed6843106de52d88333/users/3a678e305861422389a75106c379fc4c/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:53.523097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.540945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 886/1761] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:53 2022] PUT /identity/v3/projects/96a2d3d39f2a4920931044f8ee23d87b/users/78295aa21c2c4517aff1bbbb6f931f4e/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:53.544487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:53.552259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:53.748317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741a4fc-e9ba-4ac1-bcc9-86015c8a0ba5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.762897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.764417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.765056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.765931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.794268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c1db1-8d21-4219-b9e2-c613d796a1d6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:53.797621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.799158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e46ee-de0d-4e94-89c4-76ea1ebe829a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.816003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 887/1762] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:53.817674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.817970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.818240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.857569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] MFA Rules not processed for user `3a678e305861422389a75106c379fc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:53.859361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:53.909345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:53.951199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:53.952435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.953056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.953522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.954496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:53.956751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32104dc-1a20-4ffa-95f4-2b7d40e3cea1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:53.976159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 876/1763] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:53.978036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe38870-5027-4618-95ea-370d00d85a93 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:53.978403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe38870-5027-4618-95ea-370d00d85a93 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:53.978758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe38870-5027-4618-95ea-370d00d85a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:53.997586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 888/1764] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:53.998465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.025564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7fe38870-5027-4618-95ea-370d00d85a93 None None] MFA Rules not processed for user `78295aa21c2c4517aff1bbbb6f931f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:54.028061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7fe38870-5027-4618-95ea-370d00d85a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:54.036794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5dab3-3f8c-40f9-9129-5bf730e37527 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.046221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.047685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.048341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.048932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] PATH_INFO: `/v3/projects/828b7cc287854fac91cb56121745dad3/users/bca7ac0e06b345508bda7ae2d9668c93/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.078633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=828b7cc287854fac91cb56121745dad3, user_id=bca7ac0e06b345508bda7ae2d9668c93, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.083206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:54.124650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 889/1765] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:53 2022] PUT /identity/v3/projects/828b7cc287854fac91cb56121745dad3/users/bca7ac0e06b345508bda7ae2d9668c93/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:54.124650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.208588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:54.257063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 877/1766] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:54.260150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939239-1961-4886-99f3-cd7acb118503 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.261981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939239-1961-4886-99f3-cd7acb118503 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.262376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939239-1961-4886-99f3-cd7acb118503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.308428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d7360-f07e-47b5-894d-ed30ef6c4c2d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.334669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.336052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.336986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.338541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.342797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.345056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-19939239-1961-4886-99f3-cd7acb118503 None None] MFA Rules not processed for user `bca7ac0e06b345508bda7ae2d9668c93`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:54.346542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-19939239-1961-4886-99f3-cd7acb118503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:54.349135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d799c-171d-4c14-9545-5d83a71924e1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:54.460652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 878/1767] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:54.462504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939239-1961-4886-99f3-cd7acb118503 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.481595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939239-1961-4886-99f3-cd7acb118503 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.489195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.489971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.490316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.490746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.491612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.508308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:54.555574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 890/1768] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/projects => generated 373 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:54.557730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.558434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.558732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.585046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 879/1769] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:54.585046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.639398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] MFA Rules not processed for user `f07074cdff9148b581a33f12749d0277`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:54.659732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:54.660314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:54.724228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682440e0-97a1-4cb0-8f9f-4eb862d29eee admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.730483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.731178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.731468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.731798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.732553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.734813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:54.771816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 880/1770] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:54.772882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.779843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:54.879533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9676-23e0-4bac-8d5e-cb7324de8eec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.892914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.895147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.895268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 891/1771] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:54.898172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.898472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] PATH_INFO: `/v3/users/1ac6f3fd1a3a4a1495b0b329db9c2fed` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.899598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:54.900271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac6f3fd1a3a4a1495b0b329db9c2fed)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.908133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:54.948129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6601c5-247c-4191-9437-9781e05f743d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:54.958923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:54.959621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:54.959936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:54.960220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:54.981682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:54.987264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.063863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 892/1772] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:54 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:55.064942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.073506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac6f3fd1a3a4a1495b0b329db9c2fed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:55.088735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0211c7-9589-4add-bf58-73f0a05f2edc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:55.095474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac6f3fd1a3a4a1495b0b329db9c2fed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:55.097061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac6f3fd1a3a4a1495b0b329db9c2fed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:55.105623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac6f3fd1a3a4a1495b0b329db9c2fed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:55.115957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:55.119658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.133454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.133454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.133454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:55.133454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.170196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac6f3fd1a3a4a1495b0b329db9c2fed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:55.188163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 893/1773] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:55 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:55.189031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.193781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 881/1774] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:54 2022] DELETE /identity/v3/users/1ac6f3fd1a3a4a1495b0b329db9c2fed => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:55.194338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:55.194913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.199587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:55.242388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d8bff-e69a-42dc-81c7-896048bb027a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:55.334552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:55.399747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61caadd-beb0-4761-99c6-17c3b2aca0a2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:55.406541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:55.407305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.408248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.408653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.409695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:55.411681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.426680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 882/1775] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:55.427631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.429468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:55.449031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a89109-2a9d-495a-868b-bbb85bf7602b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:55.457889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:55.458720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.459339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.459848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.461467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:55.465418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:55.465907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99896b56-ae2b-40fb-b00d-9e6084a72849 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.466919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.468265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.468265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.468833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:55.480226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.483720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 883/1776] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:55.486231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.486536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.486905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.520361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:55.538512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:55.545124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 894/1777] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:01:55 2022] POST /identity/v3/users => generated 297 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:01:55.545335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:55.546185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.547942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:55.578910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 884/1778] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:55.579938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.616659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412788b9-7c66-485a-b5c9-654ad5de9f66 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:55.641657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:55.642624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:55.642890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:55.643169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] PATH_INFO: `/v3/projects/535f4390395a4734a2026545a4f8e31a/users/0d781b4a1d0c462cb08c173a6db2a2b5/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:55.710976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] RBAC: Authorizing `identity:create_grant(project_id=535f4390395a4734a2026545a4f8e31a, user_id=0d781b4a1d0c462cb08c173a6db2a2b5, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:55.727872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:55.784311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 885/1779] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:55 2022] PUT /identity/v3/projects/535f4390395a4734a2026545a4f8e31a/users/0d781b4a1d0c462cb08c173a6db2a2b5/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:55.785496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:55.815298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bff71-1d78-4db6-8db1-f923526fa220 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.048557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.050542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.051036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.051472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.054387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:56.062520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0af35c-2b5a-4494-bdea-b5ab74c5630b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.095775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.096645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.096935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.097220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] PATH_INFO: `/v3/projects/107a05c87ead480e9e58687f96e410e1/users/5aa26f9162c742aeb2be440c98bd177e/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.155103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.157513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.164790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107a05c87ead480e9e58687f96e410e1, user_id=5aa26f9162c742aeb2be440c98bd177e, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.168463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.173659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 895/1780] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:56.174832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.203493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 886/1781] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:55 2022] PUT /identity/v3/projects/107a05c87ead480e9e58687f96e410e1/users/5aa26f9162c742aeb2be440c98bd177e/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:56.204508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.213862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:56.218020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1290dd-32be-4d78-828a-01d9e2a6e737 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.317179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:56.351776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3698-d911-4850-94ff-4cbc18de4087 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.360457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.361170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.361467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.361750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.411112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.411643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.412291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.412544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.415826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.420188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.454541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 896/1782] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:56.461117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.469185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:56.493746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1ef32-5c77-4c4a-8745-3df3b40a0cdc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.508978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.511547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.513079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.513577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.517382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.519623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.548460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 897/1783] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:56.551332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.552188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.553108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.574067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 887/1784] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:56.575065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.586100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009a42-ff28-49d0-9a7c-0d37ef374aa8 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.602598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d612f3c-de9a-4e52-afa1-e7591a9bf775 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:56.608405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.611107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.611994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.613761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] PATH_INFO: `/v3/projects/a9b71f3fed884ec487b1a6599a0041a0/users/148590e13f86486ab22b61d475e49ae7/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.615862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:56.617851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:56.619176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:56.619578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:56.623286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.639488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.694232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b71f3fed884ec487b1a6599a0041a0, user_id=148590e13f86486ab22b61d475e49ae7, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:56.699260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:56.704581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 898/1785] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:56.705470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.768161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 888/1786] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:56 2022] PUT /identity/v3/projects/a9b71f3fed884ec487b1a6599a0041a0/users/148590e13f86486ab22b61d475e49ae7/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:56.769508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:56.777040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497bfb7-ad45-4c72-99d8-e883fde6f88b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:57.007009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669dd5-4e67-4d05-9881-35b5c8fa72b5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:57.020886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:57.023957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.024188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.024925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] PATH_INFO: `/v3/projects/107a05c87ead480e9e58687f96e410e1/users/5aa26f9162c742aeb2be440c98bd177e/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.030784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=107a05c87ead480e9e58687f96e410e1, user_id=5aa26f9162c742aeb2be440c98bd177e, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:57.056534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:57.106700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:57.109637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 889/1787] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:56 2022] PUT /identity/v3/projects/107a05c87ead480e9e58687f96e410e1/users/5aa26f9162c742aeb2be440c98bd177e/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:57.110627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:57.112170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.112531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.112890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] PATH_INFO: `/v3/projects/535f4390395a4734a2026545a4f8e31a/users/0d781b4a1d0c462cb08c173a6db2a2b5/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.117364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=535f4390395a4734a2026545a4f8e31a, user_id=0d781b4a1d0c462cb08c173a6db2a2b5, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:57.120130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69c05-3ca9-483f-9821-001c886d47cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:57.158229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 899/1788] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:56 2022] PUT /identity/v3/projects/535f4390395a4734a2026545a4f8e31a/users/0d781b4a1d0c462cb08c173a6db2a2b5/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:57.162230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c207d-ea5b-49a7-8f4a-ea9502197b10 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.162466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c207d-ea5b-49a7-8f4a-ea9502197b10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.162739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c207d-ea5b-49a7-8f4a-ea9502197b10 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.171528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 900/1789] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:01:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:01:57.173710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4ab9-4c8a-42a9-af04-46c4c404f64a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.174002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4ab9-4c8a-42a9-af04-46c4c404f64a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.174283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4ab9-4c8a-42a9-af04-46c4c404f64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.251157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f55b4ab9-4c8a-42a9-af04-46c4c404f64a None None] MFA Rules not processed for user `5aa26f9162c742aeb2be440c98bd177e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:57.252574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f55b4ab9-4c8a-42a9-af04-46c4c404f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:57.285112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b745fd4-db24-4044-9294-d93cf687c46c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:57.389231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:57.431452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:57.432295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.432683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.438180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.439321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:57.441864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fffc8-5889-431e-90a8-dce5114e18ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:57.453294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 901/1790] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:57.453294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.453294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.453294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.470653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 890/1791] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:01:57.470653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.476670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.477183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.517473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:57.519719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] MFA Rules not processed for user `0d781b4a1d0c462cb08c173a6db2a2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:57.522400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:57.523135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:57.557370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:57.775826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 902/1792] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:57.779081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:57.783233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:57.882730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 891/1793] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:01:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:57.885919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:57.934769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22800e7-63c6-49d8-b59c-53e628f4cdfc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:57.972472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:57.978517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:57.982480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:57.983022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] PATH_INFO: `/v3/projects/a9b71f3fed884ec487b1a6599a0041a0/users/148590e13f86486ab22b61d475e49ae7/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:57.996766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b71f3fed884ec487b1a6599a0041a0, user_id=148590e13f86486ab22b61d475e49ae7, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:58.005836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:58.030016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5eac2-c251-494d-8a52-87ecad902afa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:58.062155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:58.063174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:58.065578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 892/1794] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:01:57 2022] PUT /identity/v3/projects/a9b71f3fed884ec487b1a6599a0041a0/users/148590e13f86486ab22b61d475e49ae7/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:58.066064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:58.066509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:58.072507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:58.082838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:58.107256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:58.117366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207bd20-b221-4abb-abc9-c9f16d5f9854 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:58.158242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 903/1795] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:58.162946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:58.165422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:58.166296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:58.264943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] MFA Rules not processed for user `148590e13f86486ab22b61d475e49ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:58.269438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:58.270777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822ce4-9887-4940-b51c-d3518150a7c4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:58.282739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:58.284556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:58.284978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:58.285365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:58.477852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:58.482483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:58.486224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 904/1796] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:58.488218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:58.512931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 893/1797] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:58.529765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:58.531745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:58.569350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:58.626626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:58.794538 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cca8-7498-4532-b667-6d8adf1bea5a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:58.822518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52633-09d7-4090-b4bb-f43b181a555b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:58.823800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:58.824925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:58.825326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:58.825715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] PATH_INFO: `/v3/users/41d396bd1430439aabfcf5a6d887451f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:58.831696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d396bd1430439aabfcf5a6d887451f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:58.846087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:58.847780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:58.848080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:58.848361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:58.855870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:58.888774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:58.890499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:58.963637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 905/1798] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:58.970772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:59.010029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e23ce-5898-48b5-a666-89ef87001958 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:59.035576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:59.039949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:59.040610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:59.041848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:59.042979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d396bd1430439aabfcf5a6d887451f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:59.073796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d396bd1430439aabfcf5a6d887451f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:59.076418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d396bd1430439aabfcf5a6d887451f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:59.078371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d396bd1430439aabfcf5a6d887451f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:59.094523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:59.102156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:59.121898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7e7e024f-bc84-475a-bf27-e804723b8580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d396bd1430439aabfcf5a6d887451f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:01:59.143121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 906/1799] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:59.147314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:59.172589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 894/1800] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:01:58 2022] DELETE /identity/v3/users/41d396bd1430439aabfcf5a6d887451f => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:01:59.178143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:59.178688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:59.179049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3a8f-6f16-4596-977c-e185c9215eb2 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:59.179814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:59.250666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:01:59.256292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:59.371187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:59.371871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:59.372163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:59.372441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:59.381537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:59.443860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 895/1801] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:01:59.523114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:59.524933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:59.644424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:59.646332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:59.655281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:59.728842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 907/1802] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:59.811528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49cc2-4d9c-4735-a823-5280cf166db6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:59.821679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:59.824956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:59.825695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:59.828001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:59.865286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:59.871990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:59.896416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 896/1803] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:00.133988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.141519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.173582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64320a7-fbe8-4e14-ab16-3df604fe3193 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:00.195463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:00.196366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:00.196705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:00.196989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] PATH_INFO: `/v3/projects/4236d30e836f494792c28958d5823bb5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:00.209167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4236d30e836f494792c28958d5823bb5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:00.213634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:00.246006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.247560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.395419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4236d30e836f494792c28958d5823bb5', 'request_id': 'req-2016274f-0316-41d8-9eae-db7f1ae95f0f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.429508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4236d30e836f494792c28958d5823bb5', 'request_id': 'req-2016274f-0316-41d8-9eae-db7f1ae95f0f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.459252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4236d30e836f494792c28958d5823bb5', 'request_id': 'req-2016274f-0316-41d8-9eae-db7f1ae95f0f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.461967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:00.494245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975837c-c6da-403c-bd8c-a45a2a33bcd9 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:00.498382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] The token cache is being invalidate because project 4236d30e836f494792c28958d5823bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:00.498670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.499532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.507877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 908/1804] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:00 2022] DELETE /identity/v3/projects/4236d30e836f494792c28958d5823bb5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:00.508928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.513438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.640197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148590e13f86486ab22b61d475e49ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b71f3fed884ec487b1a6599a0041a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:00.643380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:00.643675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:00.646469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:00.647713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:00.649219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:00.653478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:00.664907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016274f-0316-41d8-9eae-db7f1ae95f0f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:00.680045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 897/1805] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:02:00.682154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:00.683834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:00.685394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:00.685742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.685916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] PATH_INFO: `/v3/users/2b3806ceb1d84c5e8fe606e442e88618` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:00.690145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3806ceb1d84c5e8fe606e442e88618)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:00.691292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.694399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:00.708078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56fb5a-da2b-4750-8407-ac1a832dbe12 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:00.716342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148590e13f86486ab22b61d475e49ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b71f3fed884ec487b1a6599a0041a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:00.719554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:00.719857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:00.720142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:00.721189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:00.725113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:00.788693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 898/1806] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:00 2022] POST /identity/v3/projects => generated 353 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:00.788693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.797127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.817583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3806ceb1d84c5e8fe606e442e88618'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.829750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3806ceb1d84c5e8fe606e442e88618'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.830610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3806ceb1d84c5e8fe606e442e88618'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.846440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3806ceb1d84c5e8fe606e442e88618'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.871416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3806ceb1d84c5e8fe606e442e88618'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:00.885879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 909/1807] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:00 2022] DELETE /identity/v3/users/2b3806ceb1d84c5e8fe606e442e88618 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:00.888375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:00.890097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:00.895860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:00.939108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621d7a2-0ade-495c-948c-530cf6811df1 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.038930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1c0d1-47ca-436e-b480-4722fecd4e83 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.050820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148590e13f86486ab22b61d475e49ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b71f3fed884ec487b1a6599a0041a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.051584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.051773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.052057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.058121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:01.058525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:01.060231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:01.064357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.065233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.065692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.065977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] PATH_INFO: `/v3/users/3a678e305861422389a75106c379fc4c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.068697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a678e305861422389a75106c379fc4c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:01.074308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:01.093287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 910/1808] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:01.164557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a678e305861422389a75106c379fc4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.174493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:01.180279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:01.189532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5fadc-6640-4bc3-af5a-84aaafeef8a5 tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.203328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a678e305861422389a75106c379fc4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.204351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a678e305861422389a75106c379fc4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.205668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a678e305861422389a75106c379fc4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.207496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148590e13f86486ab22b61d475e49ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b71f3fed884ec487b1a6599a0041a0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.208387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.209516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.209988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] PATH_INFO: `/v3/projects/ac7c8fb0c8174872b7e1e689b8d7ba6e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.211511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ac7c8fb0c8174872b7e1e689b8d7ba6e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:01.216089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:01.258947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6baf1fff-fd2f-4e82-a66f-a59da47ee445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a678e305861422389a75106c379fc4c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.293496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 899/1809] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:00 2022] DELETE /identity/v3/users/3a678e305861422389a75106c379fc4c => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:01.299963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.300254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.304992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.346704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7c8fb0c8174872b7e1e689b8d7ba6e', 'request_id': 'req-8d601f6e-5077-44ef-9983-8697a120e72e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.349506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:01.352907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.365626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7c8fb0c8174872b7e1e689b8d7ba6e', 'request_id': 'req-8d601f6e-5077-44ef-9983-8697a120e72e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.388519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7c8fb0c8174872b7e1e689b8d7ba6e', 'request_id': 'req-8d601f6e-5077-44ef-9983-8697a120e72e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.411925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] The token cache is being invalidate because project ac7c8fb0c8174872b7e1e689b8d7ba6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:01.412294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.413416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.420418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 911/1810] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:01 2022] DELETE /identity/v3/projects/ac7c8fb0c8174872b7e1e689b8d7ba6e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:01.452801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:01.454774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.454914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 900/1811] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:01.479473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601f6e-5077-44ef-9983-8697a120e72e tempest-VolumeQuotaClassesTest-1277324877 tempest-VolumeQuotaClassesTest-1277324877-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.490337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.491220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.491501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.493922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] PATH_INFO: `/v3/projects/3f83ed8823724784b406f3b5acfcb32f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.497602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f83ed8823724784b406f3b5acfcb32f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:01.499944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:01.568231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:01.584517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.593420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f83ed8823724784b406f3b5acfcb32f', 'request_id': 'req-ddd770c6-fad1-4630-aa3f-e852a5deb41a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.612577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f83ed8823724784b406f3b5acfcb32f', 'request_id': 'req-ddd770c6-fad1-4630-aa3f-e852a5deb41a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.629516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f83ed8823724784b406f3b5acfcb32f', 'request_id': 'req-ddd770c6-fad1-4630-aa3f-e852a5deb41a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.634780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:01.663546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] The token cache is being invalidate because project 3f83ed8823724784b406f3b5acfcb32f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:01.663955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.673682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ddd770c6-fad1-4630-aa3f-e852a5deb41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:01.677613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 912/1812] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:01 2022] DELETE /identity/v3/projects/3f83ed8823724784b406f3b5acfcb32f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:01.683163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.685400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.686205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.725549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:01.733599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.735200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91b56-1cff-4094-870c-a3988301f5fc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.832558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.833219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.833610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.833982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.836374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.955263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 913/1813] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:01.956184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:01.957804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:01.973454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:01.975542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ac58-078d-4def-9c10-1cbc4cae62ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:01.977042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be0b1b-b36c-4ba3-9ebc-021204be03ed service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:01.992081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:01.994193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:01.994491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:01.994780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] PATH_INFO: `/v3/users/bca7ac0e06b345508bda7ae2d9668c93` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:01.998766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca7ac0e06b345508bda7ae2d9668c93)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.002490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.006877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 901/1814] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:02.008391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:02.010217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.010541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.011970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.065286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:02.070217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.091539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 902/1815] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:02.095862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.121692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc6180-9fd8-442a-8688-b38ceed69423 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.134420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca7ac0e06b345508bda7ae2d9668c93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.143751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.144556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.144895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.145198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.146138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:02.146472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.147431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca7ac0e06b345508bda7ae2d9668c93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.148526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca7ac0e06b345508bda7ae2d9668c93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.148809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.159450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca7ac0e06b345508bda7ae2d9668c93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.164525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 903/1816] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:02.176369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.181848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd86f19e-208f-4873-835d-af59778053fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca7ac0e06b345508bda7ae2d9668c93'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.191348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865c23-b265-4daa-989f-5ff3a8e6d8f1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.192274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 914/1817] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:01 2022] DELETE /identity/v3/users/bca7ac0e06b345508bda7ae2d9668c93 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:02.196767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.197285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.197822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.229026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:02.231511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.283924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.285032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.285032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.285335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.286629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.288127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.321205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:02.359808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 904/1818] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/projects => generated 355 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:02.360907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.362632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.390363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 915/1819] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:02.421209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.423455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.509810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd320-be6a-4c5f-bd67-3da3b3452952 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.521054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.522225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.522683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.523107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.523979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.527347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.556484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 905/1820] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/users => generated 278 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:02.558325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954ec0d-d8d9-4012-b78b-5d488eb7df10 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.560246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.577541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022743-c949-4e99-9a50-02d81cb4e80b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.584069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.585636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.586357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.586852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.587921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.590667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.605348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 906/1821] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:02.616408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.630336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114f03-d182-4097-9125-5c4756be8ad2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.632905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:02.638835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.639607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.639786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.640308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.641178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.643502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.663670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 907/1822] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:02.664905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.667395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.667877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.668874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.669406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.670062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.691781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.695563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.710060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7795123-a848-4a08-bccf-d0c4c43bd432 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.724994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.726251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.726543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.737649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] PATH_INFO: `/v3/projects/33103e09bc8b47519c10937369742e0a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.739953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 916/1823] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:02.740941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:02.741321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33103e09bc8b47519c10937369742e0a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.746020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.762587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c686701-40aa-41b1-ba93-b884a7911d3c service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:02.768055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:02.769345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.769713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.770066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] PATH_INFO: `/v3/projects/d8c3423c15b34db3b5f8f15cfb0d11a1/users/81bb259fad20492181075f251be475a6/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.790733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c3423c15b34db3b5f8f15cfb0d11a1, user_id=81bb259fad20492181075f251be475a6, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:02.794609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a77f4-ab43-4c8d-a5f6-1a2d54a25086 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:02.816299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 917/1824] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:02 2022] PUT /identity/v3/projects/d8c3423c15b34db3b5f8f15cfb0d11a1/users/81bb259fad20492181075f251be475a6/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:02.825656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33103e09bc8b47519c10937369742e0a', 'request_id': 'req-a4702e4c-dcd2-4219-b8ea-359d2551537b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.827781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf4e9-28ee-4d7f-a73e-0fe906a36f13 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.828904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf4e9-28ee-4d7f-a73e-0fe906a36f13 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.829651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf4e9-28ee-4d7f-a73e-0fe906a36f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.838508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33103e09bc8b47519c10937369742e0a', 'request_id': 'req-a4702e4c-dcd2-4219-b8ea-359d2551537b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.858288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33103e09bc8b47519c10937369742e0a', 'request_id': 'req-a4702e4c-dcd2-4219-b8ea-359d2551537b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.881078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] The token cache is being invalidate because project 33103e09bc8b47519c10937369742e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:02.881426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.882313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4702e4c-dcd2-4219-b8ea-359d2551537b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:02.884492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-adbcf4e9-28ee-4d7f-a73e-0fe906a36f13 None None] MFA Rules not processed for user `81bb259fad20492181075f251be475a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:02.889502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 908/1825] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:02 2022] DELETE /identity/v3/projects/33103e09bc8b47519c10937369742e0a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:02.890036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adbcf4e9-28ee-4d7f-a73e-0fe906a36f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.891690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.892092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.892442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:02.925367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:02.926616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:02.973609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:02.986954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 918/1826] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:02.989418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e1e7-5abc-4173-b6de-29a05436395b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:02.989992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e1e7-5abc-4173-b6de-29a05436395b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:02.990674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e1e7-5abc-4173-b6de-29a05436395b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.013212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6808e1e7-5abc-4173-b6de-29a05436395b None None] MFA Rules not processed for user `f07074cdff9148b581a33f12749d0277`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:03.014732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6808e1e7-5abc-4173-b6de-29a05436395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:03.034656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 909/1827] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:03.035701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.039097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:03.056951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d162cba-8aad-45d1-8988-6c626b2d83ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.062759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.063510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.063800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.064083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.065465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.067706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.104186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 919/1828] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:03.104186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.104186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.104186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.134633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 910/1829] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/projects => generated 357 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:03.137666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.152013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:03.153527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:03.231845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:03.265142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 920/1830] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:03.266174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.275979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872262-fd2a-4c09-8253-2ecc53ba87cd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.277062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9fbe1-8ae8-4321-882c-53cd8d4d3daf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.282265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.282959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.283250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.283438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.283553 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.284083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.284329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:03.284428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.284664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.284771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.285377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.286278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.287416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.301973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 921/1831] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:03.305016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.316247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923728a-ea6a-4432-93a2-827731f7c649 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.321923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.322607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.322899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.323804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.324792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.326041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 911/1832] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:03.327550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.332043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.352548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8a4b-f1e4-47c1-a6f8-e680eaa16eae admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.359756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.360868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.361360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.361807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.363457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.365616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.372634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 922/1833] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:03.375270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.379160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 912/1834] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:03.382922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.396180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980bc9d-f286-401d-b58d-11ff4f6b6e47 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.472032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ff88-7562-495f-8e35-728c2988f76f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.478338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.479047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.479400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.479686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.480400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.483142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.510431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.512735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.513715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.514146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] PATH_INFO: `/v3/projects/a63d7fc04dc94028b55362bdeece48f7/users/cb1e9a5138c64c2f83bd47718b4da99b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.515381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 923/1835] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:03.517667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.542218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63d7fc04dc94028b55362bdeece48f7, user_id=cb1e9a5138c64c2f83bd47718b4da99b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.542661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd676b-0832-4477-8a13-233ab602d7fa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.545174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.558874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.559827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.560204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.560579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.561644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.563362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.576599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 913/1836] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:03 2022] PUT /identity/v3/projects/a63d7fc04dc94028b55362bdeece48f7/users/cb1e9a5138c64c2f83bd47718b4da99b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:03.579418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.581496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 924/1837] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:03.585415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.599978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840221-6673-4a2e-a9e0-b1c5abc55d8f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.612150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:03.673948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019562b-9f55-4618-b6cc-fd22aa2bcb11 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.679037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.679795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.680143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.680485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.681347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.684226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.686382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.688263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.688984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.689428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.690315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.693433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.713201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 925/1838] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:03.714681 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.731209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 914/1839] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:03.733211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:03.740183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ea7f-25e6-4825-b125-84192b25f6ad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.747355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.748331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.748657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.749037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] PATH_INFO: `/v3/projects/274345e0b3444d5792837f2d490f66d8/users/32161b154f4f411cb25da9f4ad316062/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.749506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98f739-0ea8-4fbc-84bc-3f93a6b29dc7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:03.761649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:03.762817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.763250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.763743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] PATH_INFO: `/v3/projects/a63d7fc04dc94028b55362bdeece48f7/users/cb1e9a5138c64c2f83bd47718b4da99b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.767747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63d7fc04dc94028b55362bdeece48f7, user_id=cb1e9a5138c64c2f83bd47718b4da99b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.773009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=274345e0b3444d5792837f2d490f66d8, user_id=32161b154f4f411cb25da9f4ad316062, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:03.774488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ddf3-5d0f-4ec1-b44e-cf85cac4c31d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.777037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6456d4-3eee-46ef-8a0c-efc68288e4fb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:03.797240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 915/1840] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:03 2022] PUT /identity/v3/projects/a63d7fc04dc94028b55362bdeece48f7/users/cb1e9a5138c64c2f83bd47718b4da99b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:03.809369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 926/1841] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:03 2022] PUT /identity/v3/projects/274345e0b3444d5792837f2d490f66d8/users/32161b154f4f411cb25da9f4ad316062/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:03.811133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.811517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.811812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.816924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:03.817355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:03.817585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:03.827070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:03.852545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] MFA Rules not processed for user `32161b154f4f411cb25da9f4ad316062`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:03.853920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:03.917948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] MFA Rules not processed for user `cb1e9a5138c64c2f83bd47718b4da99b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:03.919804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:03.971965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 927/1842] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:03.973138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.087399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22cd9-ccca-40cd-a5ed-b200bb3a46ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.096087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.096897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.097200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.097646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.099156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.101907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.154493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 916/1843] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:04.155666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.157323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.178007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c7e5d-7440-4c72-bc1f-20930d3264c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.186720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.187570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.187948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.188351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] PATH_INFO: `/v3/projects/828b7cc287854fac91cb56121745dad3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.190442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828b7cc287854fac91cb56121745dad3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.192792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.204153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 928/1844] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:03 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:04.205671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.207496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.289968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828b7cc287854fac91cb56121745dad3', 'request_id': 'req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.313118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828b7cc287854fac91cb56121745dad3', 'request_id': 'req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.336487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828b7cc287854fac91cb56121745dad3', 'request_id': 'req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.359656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68f6d-9af3-4bff-8116-2b85101ed5e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.360212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] The token cache is being invalidate because project 828b7cc287854fac91cb56121745dad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:04.360503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.361206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.363008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 917/1845] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:04 2022] DELETE /identity/v3/projects/828b7cc287854fac91cb56121745dad3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:04.363782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.366988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.397473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:04.420087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:04.450979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.452762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.453137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.453700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.456544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.533347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7c571-6f96-4feb-82e8-1a4f8d0097a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.545995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.546892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.547183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.547463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] PATH_INFO: `/v3/projects/1fbb080b357b4ed6843106de52d88333` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.551201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbb080b357b4ed6843106de52d88333)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.553240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.575132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.576793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.605070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 929/1846] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:04 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:04.609753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.611175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.641167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4048-b1aa-4f36-a5d4-c57779b9a28c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.654417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.655118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.656032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.656327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.657073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.659811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb053c-dca5-4ec4-80cc-b0155115acd1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.695499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 930/1847] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:04 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:04.697036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.697330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.697612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.701286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 931/1848] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:02:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:02:04.702716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.706805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.725031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbb080b357b4ed6843106de52d88333', 'request_id': 'req-a740e17e-c734-4006-8e2a-cdb19b04d9ff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.748247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbb080b357b4ed6843106de52d88333', 'request_id': 'req-a740e17e-c734-4006-8e2a-cdb19b04d9ff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.762618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbb080b357b4ed6843106de52d88333', 'request_id': 'req-a740e17e-c734-4006-8e2a-cdb19b04d9ff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.790360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] The token cache is being invalidate because project 1fbb080b357b4ed6843106de52d88333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:04.794327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.795629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.804163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 918/1849] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:04 2022] DELETE /identity/v3/projects/1fbb080b357b4ed6843106de52d88333 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:04.823107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:04.833107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.835595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08872d-2feb-4a4a-8749-9a0cd186209c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.841007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.842276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.842276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.842470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] PATH_INFO: `/v3/users/148590e13f86486ab22b61d475e49ae7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.845490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148590e13f86486ab22b61d475e49ae7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.847438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.859737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e17e-c734-4006-8e2a-cdb19b04d9ff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:04.865265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:04.866216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.866593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.866963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.867898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:04.871120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb442-6b6d-4e89-934c-6701aebc1cd6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:04.909763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 919/1850] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:04.911523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:04.913215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:04.913776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:04.957236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:04.958455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:04.981612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148590e13f86486ab22b61d475e49ae7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.992118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148590e13f86486ab22b61d475e49ae7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:04.993051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148590e13f86486ab22b61d475e49ae7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.001965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148590e13f86486ab22b61d475e49ae7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.011299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:05.024466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148590e13f86486ab22b61d475e49ae7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.040716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 932/1851] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:04 2022] DELETE /identity/v3/users/148590e13f86486ab22b61d475e49ae7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:05.054137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.058059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.064020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 920/1852] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:02:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:05.065551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.067927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.086320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb47c99-1cb9-490a-b3cb-f29902172662 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.112688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:05.202415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.203713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.204194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.204577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] PATH_INFO: `/v3/users/0d781b4a1d0c462cb08c173a6db2a2b5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.206116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d781b4a1d0c462cb08c173a6db2a2b5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.209455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.210427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce165db-955b-4877-be99-491aa53f5365 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.222920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.223628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.223997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.224284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.227234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.286971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d781b4a1d0c462cb08c173a6db2a2b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.300122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d781b4a1d0c462cb08c173a6db2a2b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.300954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d781b4a1d0c462cb08c173a6db2a2b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.301583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d781b4a1d0c462cb08c173a6db2a2b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.305693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.307676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.317710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d781b4a1d0c462cb08c173a6db2a2b5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.320177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 933/1853] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:05 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:05.321180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.322806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.341265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e0117-33a4-472c-acc9-93871eab709c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.346577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.347260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.347540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.347813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.348721 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.351223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.353737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 921/1854] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:05 2022] DELETE /identity/v3/users/0d781b4a1d0c462cb08c173a6db2a2b5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:05.357784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.374075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 934/1855] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:05.375768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.379033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.387013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.412333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03927bf-cd35-4d47-9649-36e9dd76a763 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.506342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.507373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.507670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.507958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] PATH_INFO: `/v3/projects/013ead7effa04b91a6ebda4b48b757c9/users/b97b772ed6494a2999e485a873f438bc/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.521034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3cd-0991-47d8-b002-a59c2086f4e2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.529637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013ead7effa04b91a6ebda4b48b757c9, user_id=b97b772ed6494a2999e485a873f438bc, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.530154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.530948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.531512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.531998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.532610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.534892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.551827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 935/1856] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:05 2022] PUT /identity/v3/projects/013ead7effa04b91a6ebda4b48b757c9/users/b97b772ed6494a2999e485a873f438bc/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:05.564133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.567895 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.570540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.575590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f737af-b2b3-4120-97d5-5fa6afe42663 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.592327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:05.596731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 922/1857] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:05.600533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.601496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.601804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.639413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] MFA Rules not processed for user `b97b772ed6494a2999e485a873f438bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:05.641893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.666199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be5f99-ef5d-4412-a130-3b2c4abd2137 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.671193 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:05.672039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.673783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.674166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.674546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] PATH_INFO: `/v3/projects/a9b71f3fed884ec487b1a6599a0041a0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.676314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b71f3fed884ec487b1a6599a0041a0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.680562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.755682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b71f3fed884ec487b1a6599a0041a0', 'request_id': 'req-d0ac2cc9-88be-4067-b157-589ec46cc949'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.760219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 923/1858] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:05.762055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.767853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b71f3fed884ec487b1a6599a0041a0', 'request_id': 'req-d0ac2cc9-88be-4067-b157-589ec46cc949'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.772686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbe696-6c2d-419a-9c15-ec32d08f05fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.778779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b71f3fed884ec487b1a6599a0041a0', 'request_id': 'req-d0ac2cc9-88be-4067-b157-589ec46cc949'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.794015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] The token cache is being invalidate because project a9b71f3fed884ec487b1a6599a0041a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:05.794431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.795582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0ac2cc9-88be-4067-b157-589ec46cc949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:05.798321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 936/1859] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:05 2022] DELETE /identity/v3/projects/a9b71f3fed884ec487b1a6599a0041a0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:05.805880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.806128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.806518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.827833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:05.829201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.843831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.844824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.845256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.845539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.846300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.848420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.880434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 924/1860] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:05 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:05.885166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.890962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.905176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 937/1861] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:05.906126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:05.907603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:05.921095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd35fa-514a-4846-b943-8d60a690ed13 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.971975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f3c0-242e-4429-a746-552f4b5c655f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:05.975119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:05.980859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.981876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.983019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.983637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:05.984537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:05.987148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7ef1-1a9f-4762-83b2-bcab23ccf74c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:05.995622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:05.997443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:05.998130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:05.998876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] PATH_INFO: `/v3/projects/535f4390395a4734a2026545a4f8e31a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.001414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=535f4390395a4734a2026545a4f8e31a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.004768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.014717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 925/1862] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:05 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:06.016390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.016861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.017266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.040934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:06.042227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.054345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 926/1863] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:06.054345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.064277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee5768-6d7a-481b-ad96-22e95067a95f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.070052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.070804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.071301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.071838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.073004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.075860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.088680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 927/1864] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:06.089161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.099676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50c26d-0afe-4812-a7cb-67d32f20993b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.100131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535f4390395a4734a2026545a4f8e31a', 'request_id': 'req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:06.110189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:06.113321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535f4390395a4734a2026545a4f8e31a', 'request_id': 'req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:06.129994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535f4390395a4734a2026545a4f8e31a', 'request_id': 'req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:06.143482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] The token cache is being invalidate because project 535f4390395a4734a2026545a4f8e31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:06.144114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:06.145013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:06.147046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 938/1865] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:05 2022] DELETE /identity/v3/projects/535f4390395a4734a2026545a4f8e31a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:06.147821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.149247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.165395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.166098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.166379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.166691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.167448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:06.167758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.169454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929aca7-2cd0-4d81-87f9-604aec7c2b78 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.182768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 928/1866] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:06.184113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.184404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.184706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.206931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:06.208307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.216804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 929/1867] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:06.217735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.218980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.231098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce240035-3b10-40fd-967b-9b98a872f10a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.235969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.236611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.236931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.237210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.237893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.239807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa4b7-2386-4cf9-ace2-d6b92ab96d87 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.243827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2d227-6300-4ff9-a16b-4fe4be4dfbcf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.249703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.250624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.250907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.251181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] PATH_INFO: `/v3/projects/d9452f9d85a74840acce55aee548c50e/users/3aaa612dbcf84409b865ddf094464cf3/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.266578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 930/1868] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:06.268515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.268515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.268839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.271407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9452f9d85a74840acce55aee548c50e, user_id=3aaa612dbcf84409b865ddf094464cf3, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.274208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.288440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 939/1869] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:06 2022] PUT /identity/v3/projects/d9452f9d85a74840acce55aee548c50e/users/3aaa612dbcf84409b865ddf094464cf3/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:06.291318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.291415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:06.291415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.351718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 931/1870] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:06.352869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.357937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931bc6d-afeb-4d56-8580-3e2d1514d3ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.362793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d62e1-d2cb-4b48-b982-453d4e310a42 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.363034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.363779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.364143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.364497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.365326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.367351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.367475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.367994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.368275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.368552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.369333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.373745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.376735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:06.394349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 932/1871] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:06.395282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.416168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78f335-6ab5-4dfe-9e74-74def1269eba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.422148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.422901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.423012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.423256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.423444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 940/1872] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:06.423986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:06.424288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.426186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.427173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.444335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 933/1873] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:06.445655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.449464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e1959-b9f4-432c-bb07-48b3b8ffe4cc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.456029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.457235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.457607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.457963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] PATH_INFO: `/v3/projects/d9452f9d85a74840acce55aee548c50e/users/3aaa612dbcf84409b865ddf094464cf3/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.460645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9452f9d85a74840acce55aee548c50e, user_id=3aaa612dbcf84409b865ddf094464cf3, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.464590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.466772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d6466-7513-424e-8e7a-267daa8a1eb0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.477106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.478506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.480805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.481341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.482217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.484342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bde0d-4c0f-4b23-8d3a-bdab36207380 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.489055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 941/1874] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:06 2022] PUT /identity/v3/projects/d9452f9d85a74840acce55aee548c50e/users/3aaa612dbcf84409b865ddf094464cf3/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:06.490017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.503952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 934/1875] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:06.507060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.507619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.508401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.574429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] MFA Rules not processed for user `3aaa612dbcf84409b865ddf094464cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:06.576190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.622439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757521a3-df46-4c10-ac1b-98e04fa80b4d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.635439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.636268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.638767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.638890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.639663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.645188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.653399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:06.735965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 942/1876] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:06.739092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.739939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 935/1877] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:06.741428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.767501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7cf1e-f68f-4851-882e-1316f4e7d4fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.842140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0b9a-eb30-46ef-8b5d-422a43ce0b21 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.865339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.866087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.866315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.866589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.869529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.875634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.886690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.887583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.887972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.888980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.889447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.892130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:06.913627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 943/1878] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:06.913627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.917634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:06.922343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 936/1879] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:06 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:06.927694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:06.956129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cf312-8577-4fb9-8140-6f79b7d1ccf4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:06.970347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:06.971326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:06.971695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:06.971977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] PATH_INFO: `/v3/projects/9af15900c48c427ba34e1957f0673872/users/ccbce68e2ba547929eb74838701f1563/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:06.994656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9af15900c48c427ba34e1957f0673872, user_id=ccbce68e2ba547929eb74838701f1563, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:06.999264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.046886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 937/1880] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:06 2022] PUT /identity/v3/projects/9af15900c48c427ba34e1957f0673872/users/ccbce68e2ba547929eb74838701f1563/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:07.046886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.073828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81d4d-4167-41f3-8f26-af9e7f6aa4f0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.089840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:07.188353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6169cde-d46a-49b0-bed6-f75b4ca88834 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.195475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.196571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.197103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.197560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.199237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.201578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94697900-f041-4ed3-9a5e-751e713e37b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.217010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 938/1881] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:07.218561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.219124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.219553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.278523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.280608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.283547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.284137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.318398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] MFA Rules not processed for user `ccbce68e2ba547929eb74838701f1563`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:07.323489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:07.365158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:07.442374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 939/1882] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:07.443802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.459203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82bc40-24de-44cb-b42c-c4edbe2fbb0f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.466476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.468677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.469151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.469588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.470059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.471747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.473594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.474542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.495269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 940/1883] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:07.496374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.505678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 944/1884] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:07.506914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.532450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87867cc7-2632-4c55-b2d4-4977c51100d7 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.535103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f01e7-ff30-42de-b77e-0d77e04f92ae admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.545177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.547164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.548884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.549285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] PATH_INFO: `/v3/projects/e7b034ee85ae425dbdad0139dfb1901d/users/e8238738e1b24646a3d3d1272ffb0536/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.550106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.550581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.552074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.552372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.553128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.555129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.594692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b034ee85ae425dbdad0139dfb1901d, user_id=e8238738e1b24646a3d3d1272ffb0536, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.603332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5f9c-cf5d-4686-b193-681f54c63f7d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.638907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 945/1885] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:07 2022] PUT /identity/v3/projects/e7b034ee85ae425dbdad0139dfb1901d/users/e8238738e1b24646a3d3d1272ffb0536/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:07.650806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.651829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.652529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.669881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 941/1886] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:07 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:07.671471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.731792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] MFA Rules not processed for user `e8238738e1b24646a3d3d1272ffb0536`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:07.733038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:07.899140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 946/1887] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:07.902953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e6036-8ab5-4071-ba54-1cccfe62b273 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.904728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.915339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.916045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.916329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.916734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.918461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.920427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.929384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ddd9e-14fd-4221-bf90-36f43aa1cb59 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.938821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.940864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.942845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.943084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.945030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.950880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:07.951851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 942/1888] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:07 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:07.955767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:07.970355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d5a11-7563-48c3-ac72-ecc3f0798010 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:07.978573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:07.981294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:07.983546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:07.984087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:07.986814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:07.990209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.020311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 947/1889] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:07 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:08.020487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 943/1890] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:08.028391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.028494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.045123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:08.055893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f412ab-04de-486c-8f7f-98fc28f497d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.166181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c7b7-b403-4533-943e-211e1df2ff29 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.171574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.172408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.172851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.173143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.174858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.176498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.221336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 948/1891] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:08.223797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.240916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.242938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.243785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.245677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.245938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d936a-5141-49b9-8686-3d8abe428f60 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.247468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.253297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.259259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.260751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.261156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.261575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] PATH_INFO: `/v3/projects/9adc8b67c9ac441c8dff07d6800fde51/users/c49f83ed62cc4d85b52972b367f03a3e/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.284957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:08.304236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 944/1892] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:08 2022] POST /identity/v3/users => generated 299 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:08.306019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9adc8b67c9ac441c8dff07d6800fde51, user_id=c49f83ed62cc4d85b52972b367f03a3e, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.308967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.314023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72105b0-72f4-4760-b9c0-0d0f5ca80284 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.330586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810eb665-f467-4eb6-8acf-c048cd0e3a68 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.336667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 949/1893] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:08 2022] PUT /identity/v3/projects/9adc8b67c9ac441c8dff07d6800fde51/users/c49f83ed62cc4d85b52972b367f03a3e/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:08.349008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.349777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.350423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.418419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] MFA Rules not processed for user `c49f83ed62cc4d85b52972b367f03a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:08.424168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:08.469561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.470279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.470564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.470881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.475382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.478051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.511019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 945/1894] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:08.514215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.537572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5455-2dae-473e-9a92-ab9797cb111a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.563362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.564446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.565098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.566864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] PATH_INFO: `/v3/projects/846dc5fbc6cb4269b80e3147a3584660/users/86f5a715d840469289a65d4b2dfde272/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.605570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=846dc5fbc6cb4269b80e3147a3584660, user_id=86f5a715d840469289a65d4b2dfde272, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.609744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.616937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 950/1895] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:08.618085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.655519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 946/1896] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:08 2022] PUT /identity/v3/projects/846dc5fbc6cb4269b80e3147a3584660/users/86f5a715d840469289a65d4b2dfde272/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:08.657505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ed46e-b165-4d7a-a169-c29d5cb2a3d3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.660416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.745852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d92ff-defa-44cb-a171-388e31ccf6ab admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.753653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.754533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.755257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.755688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.756651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.758493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.780070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 947/1897] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:08.781587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.796751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924fa26-aa71-4596-af82-a1b2394c5a5a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:08.801446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.805000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.807013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.807199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] PATH_INFO: `/v3/projects/846dc5fbc6cb4269b80e3147a3584660/users/86f5a715d840469289a65d4b2dfde272/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.814815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] RBAC: Authorizing `identity:create_grant(project_id=846dc5fbc6cb4269b80e3147a3584660, user_id=86f5a715d840469289a65d4b2dfde272, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.820533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a245b-d6d9-409e-a060-09d0df39027e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.871097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 948/1898] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:08 2022] PUT /identity/v3/projects/846dc5fbc6cb4269b80e3147a3584660/users/86f5a715d840469289a65d4b2dfde272/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:08.874904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.875320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.875832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.880173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:08.882206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:08.883315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:08.883909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:08.887099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:08.891559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:08.922438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:08.927339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] MFA Rules not processed for user `86f5a715d840469289a65d4b2dfde272`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:08.929079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:08.948223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 951/1899] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:08 2022] POST /identity/v3/projects => generated 377 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:08.953149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:08.995087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:09.035836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 949/1900] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:09.043102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.103348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4e76b-e544-4c49-8275-369bc2c3665e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.109903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.110620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.111029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.111363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.113115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.115856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.140804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007036-ea5f-46b6-8c9f-45fb2dbe32d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.163248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 952/1901] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:08 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:09.165111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.169975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:09.237809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.239152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.239152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.239319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.241508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:09.262628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3214874-3d19-4676-a8b1-a4d12f0ec20a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.268229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.268929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.269212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.269492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.332026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.333899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.345634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 950/1902] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:09.346543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.358744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.359620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6d50-3c1d-42aa-9ba6-7a630a892aaa service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.360199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.365149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.365908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.366271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.366624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.367439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.373028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.374048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 953/1903] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:09.391117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 951/1904] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:09.394796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.410301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee35a-aa72-4c45-9359-64d7c261b051 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.415971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.417565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.418090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.418090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] PATH_INFO: `/v3/projects/72294d2b96c44b37877b0804980578c7/users/2d14843a17174247bb44fe3caa01e14b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.447806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72294d2b96c44b37877b0804980578c7, user_id=2d14843a17174247bb44fe3caa01e14b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.451550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.472924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 954/1905] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:09 2022] PUT /identity/v3/projects/72294d2b96c44b37877b0804980578c7/users/2d14843a17174247bb44fe3caa01e14b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:09.477632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.491643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fe9a-4970-40d7-ac4e-7fa039c5afe9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.539778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:09.575016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.575807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.576128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.576452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.577856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.579511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806b4b8-add6-4366-8a10-1a715cac54f1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.598964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 952/1906] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:09.602267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:09.613762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f883-58ae-4a6d-90e2-794bd9f192b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:09.625916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:09.628122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.628814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.629302 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] PATH_INFO: `/v3/projects/72294d2b96c44b37877b0804980578c7/users/2d14843a17174247bb44fe3caa01e14b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.632219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72294d2b96c44b37877b0804980578c7, user_id=2d14843a17174247bb44fe3caa01e14b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:09.636757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab66de-7ed4-4b69-a137-6386a6eb305e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:09.646698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:09.655784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 955/1907] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:09 2022] PUT /identity/v3/projects/72294d2b96c44b37877b0804980578c7/users/2d14843a17174247bb44fe3caa01e14b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:09.667232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82794e1-b883-4d60-b551-d5367debebac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.667681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82794e1-b883-4d60-b551-d5367debebac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.668161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82794e1-b883-4d60-b551-d5367debebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.726076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e82794e1-b883-4d60-b551-d5367debebac None None] MFA Rules not processed for user `2d14843a17174247bb44fe3caa01e14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:09.728045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e82794e1-b883-4d60-b551-d5367debebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:09.923207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ef5f-c9e0-42b7-be00-f711f76131a5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.923702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ef5f-c9e0-42b7-be00-f711f76131a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.924048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ef5f-c9e0-42b7-be00-f711f76131a5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.925691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 956/1908] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:02:09.946252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:09.946252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:09.946448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:09.952653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 953/1909] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:09.985730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:09.986970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:10.135676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 957/1910] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:02:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:10.216846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82794e1-b883-4d60-b551-d5367debebac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:10.304186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82794e1-b883-4d60-b551-d5367debebac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:10.311032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:10.314541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:10.314949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:10.315620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:10.318422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:10.319901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:10.335660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cbf72-3ffe-44b1-9ac1-4b082378b767 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:10.347712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:10.348493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:10.348913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:10.349526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:10.361607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:10.363576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:10.382992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 958/1911] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:10.432224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:10.434852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:10.449082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 954/1912] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:12.427485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:12.436778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0f50-4047-4341-9e55-73274ee9b294 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:12.440364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:12.441088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:12.441366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:12.441643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:12.503216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:12.504678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:12.514498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 959/1913] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:13.518733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:13.528215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692052e-d4d0-4aa5-99c8-c3a29aab4838 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:13.535746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:13.536411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:13.536859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:13.537022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:13.554066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:13.555689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:13.562189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:13.570372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 955/1914] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:14.401902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:14.419052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff1462-10c0-4cd5-bc86-f20c4ebd3ee8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:14.425686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:14.426637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:14.427182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:14.427656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:14.430869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:14.522640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:14.524598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:14.547837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 960/1915] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:14.853688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:14.865596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a111b-2a3b-4ee5-acdb-7d732d75c1bc service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:14.872334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:14.873717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:14.874296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:14.875465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:14.966576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:14.968195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201aa7a7-ceee-41c7-9664-561f5d166215 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:14.981202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 956/1916] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:17.357952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:17.382040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:17.459526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452142e-1d51-467f-8815-575aff415b26 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:17.465270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:17.466185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:17.466508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:17.466749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] PATH_INFO: `/v3/users/32161b154f4f411cb25da9f4ad316062` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:17.468390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=32161b154f4f411cb25da9f4ad316062)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:17.470616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:17.538327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32161b154f4f411cb25da9f4ad316062'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:17.549578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32161b154f4f411cb25da9f4ad316062'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:17.550364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32161b154f4f411cb25da9f4ad316062'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:17.559692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32161b154f4f411cb25da9f4ad316062'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:17.575746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32161b154f4f411cb25da9f4ad316062'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:17.583952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 961/1917] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:17 2022] DELETE /identity/v3/users/32161b154f4f411cb25da9f4ad316062 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:17.589658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:17.590239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:17.590239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:17.611375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:17.612654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:17.702159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 957/1918] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:17.709087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:17.710931 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:17.804576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972264-f0d9-4310-9223-854bcf5f4dfe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:17.809031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:17.809785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:17.810150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:17.810477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:17.823866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:17.825641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:17.840670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 962/1919] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:18.349033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:18.350522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.371899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:18.373886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.374789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90278d3f-ad1c-4563-876c-ae26c5a93bc2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:18.380919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:18.381799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:18.382095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:18.382375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] PATH_INFO: `/v3/projects/274345e0b3444d5792837f2d490f66d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:18.383611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=274345e0b3444d5792837f2d490f66d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:18.385779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:18.395352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3359-31e1-46da-aa6b-1fba9b764abd service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:18.400489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:18.401486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:18.402019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:18.402309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] PATH_INFO: `/v3/users/ccbce68e2ba547929eb74838701f1563` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:18.403648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbce68e2ba547929eb74838701f1563)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:18.405682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:18.450733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274345e0b3444d5792837f2d490f66d8', 'request_id': 'req-10a693ea-8353-4595-bf19-b1995ff93bad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.457805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274345e0b3444d5792837f2d490f66d8', 'request_id': 'req-10a693ea-8353-4595-bf19-b1995ff93bad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.458723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:18.466283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274345e0b3444d5792837f2d490f66d8', 'request_id': 'req-10a693ea-8353-4595-bf19-b1995ff93bad'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.473858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbce68e2ba547929eb74838701f1563'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.477703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] The token cache is being invalidate because project 274345e0b3444d5792837f2d490f66d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:18.478017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.478675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.481122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 958/1920] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:18 2022] DELETE /identity/v3/projects/274345e0b3444d5792837f2d490f66d8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:18.483341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbce68e2ba547929eb74838701f1563'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.483995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbce68e2ba547929eb74838701f1563'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.485437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:18.486776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.491511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbce68e2ba547929eb74838701f1563'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.507099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:18.515361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4a8119a5-b25b-4c50-90cc-78cc9706aac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbce68e2ba547929eb74838701f1563'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.523656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 963/1921] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:18 2022] DELETE /identity/v3/users/ccbce68e2ba547929eb74838701f1563 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:18.528876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7740-a601-497d-b384-8c712b826068 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:18.529292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7740-a601-497d-b384-8c712b826068 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:18.529559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae7740-a601-497d-b384-8c712b826068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:18.560092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0fae7740-a601-497d-b384-8c712b826068 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:18.561360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0fae7740-a601-497d-b384-8c712b826068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.573223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a693ea-8353-4595-bf19-b1995ff93bad admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:18.626140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 964/1922] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:18.636691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae7740-a601-497d-b384-8c712b826068 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:18.638008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0fae7740-a601-497d-b384-8c712b826068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.638433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:18.639328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:18.639620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:18.639928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] PATH_INFO: `/v3/users/b97b772ed6494a2999e485a873f438bc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:18.641296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97b772ed6494a2999e485a873f438bc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:18.643603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:18.693164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97b772ed6494a2999e485a873f438bc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.696580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae7740-a601-497d-b384-8c712b826068 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:18.699908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97b772ed6494a2999e485a873f438bc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.700535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:18.700700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97b772ed6494a2999e485a873f438bc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.701223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:18.701337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97b772ed6494a2999e485a873f438bc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.701508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:18.701789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:18.703576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:18.714957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97b772ed6494a2999e485a873f438bc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:18.716598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:18.718082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:18.728552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 965/1923] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:18.729834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 959/1924] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:18 2022] DELETE /identity/v3/users/b97b772ed6494a2999e485a873f438bc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:19.112160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:19.113571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:19.132290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad1c82-48fb-40f4-aa31-b750eaae56d2 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:19.189341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:19.192981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:19.247809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:19.298557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:19.299699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:19.299984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:19.300256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] PATH_INFO: `/v3/projects/013ead7effa04b91a6ebda4b48b757c9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:19.303465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013ead7effa04b91a6ebda4b48b757c9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:19.305907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:19.318402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:19.370175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715e0e-9415-4cc7-b72b-ffadd4a25eed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:19.380530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:19.381481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:19.382923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:19.383214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] PATH_INFO: `/v3/projects/9af15900c48c427ba34e1957f0673872` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:19.388911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9af15900c48c427ba34e1957f0673872)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:19.391835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:19.434114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013ead7effa04b91a6ebda4b48b757c9', 'request_id': 'req-acc0375c-663a-4ae6-96ef-14d0370579e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.442530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013ead7effa04b91a6ebda4b48b757c9', 'request_id': 'req-acc0375c-663a-4ae6-96ef-14d0370579e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.453296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013ead7effa04b91a6ebda4b48b757c9', 'request_id': 'req-acc0375c-663a-4ae6-96ef-14d0370579e4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.473983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9af15900c48c427ba34e1957f0673872', 'request_id': 'req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.480538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] The token cache is being invalidate because project 013ead7effa04b91a6ebda4b48b757c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:19.480995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.481844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9af15900c48c427ba34e1957f0673872', 'request_id': 'req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.483495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.487049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 966/1925] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:19 2022] DELETE /identity/v3/projects/013ead7effa04b91a6ebda4b48b757c9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:19.491492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:19.493889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:19.498406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9af15900c48c427ba34e1957f0673872', 'request_id': 'req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.515651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] The token cache is being invalidate because project 9af15900c48c427ba34e1957f0673872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:19.516399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.518181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.520547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 960/1926] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:19 2022] DELETE /identity/v3/projects/9af15900c48c427ba34e1957f0673872 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:19.524544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:19.526814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:19.584427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0375c-663a-4ae6-96ef-14d0370579e4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:19.617705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aff3a3-1345-42e3-a0ff-cca7e3d4a434 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:19.624219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:19.625355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:19.625756 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:19.626424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] PATH_INFO: `/v3/users/c49f83ed62cc4d85b52972b367f03a3e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:19.627819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49f83ed62cc4d85b52972b367f03a3e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:19.630224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:19.668851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:19.669837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:19.670256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:19.670556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] PATH_INFO: `/v3/users/3aaa612dbcf84409b865ddf094464cf3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:19.672100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aaa612dbcf84409b865ddf094464cf3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:19.674509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:19.692359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49f83ed62cc4d85b52972b367f03a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.702090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49f83ed62cc4d85b52972b367f03a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.703457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49f83ed62cc4d85b52972b367f03a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.704145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49f83ed62cc4d85b52972b367f03a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.719311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49f83ed62cc4d85b52972b367f03a3e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.719532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:19.727493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:19.742469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 961/1927] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:19 2022] DELETE /identity/v3/users/c49f83ed62cc4d85b52972b367f03a3e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:19.742659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aaa612dbcf84409b865ddf094464cf3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.758273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aaa612dbcf84409b865ddf094464cf3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.759078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aaa612dbcf84409b865ddf094464cf3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.768941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aaa612dbcf84409b865ddf094464cf3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.783998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aaa612dbcf84409b865ddf094464cf3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:19.791770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 967/1928] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:19 2022] DELETE /identity/v3/users/3aaa612dbcf84409b865ddf094464cf3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:20.269469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:20.271395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.333273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:20.335662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.395397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e8532-b8a1-43ae-8d34-51ffd91bfcbf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:20.423024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b286-798b-40ee-bc06-93d95ab6d579 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:20.428487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:20.429530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.429815 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.430372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] PATH_INFO: `/v3/projects/d9452f9d85a74840acce55aee548c50e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.431937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9452f9d85a74840acce55aee548c50e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:20.434180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:20.483478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:20.484500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.484815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.485089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] PATH_INFO: `/v3/projects/9adc8b67c9ac441c8dff07d6800fde51` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.486414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9adc8b67c9ac441c8dff07d6800fde51)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:20.488353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:20.501110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9452f9d85a74840acce55aee548c50e', 'request_id': 'req-a6d9e68c-7936-4640-95ef-bd9ec2224585'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.507309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9452f9d85a74840acce55aee548c50e', 'request_id': 'req-a6d9e68c-7936-4640-95ef-bd9ec2224585'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.518243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9452f9d85a74840acce55aee548c50e', 'request_id': 'req-a6d9e68c-7936-4640-95ef-bd9ec2224585'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.530643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] The token cache is being invalidate because project d9452f9d85a74840acce55aee548c50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:20.530950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.531601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.537529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 968/1929] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:20 2022] DELETE /identity/v3/projects/d9452f9d85a74840acce55aee548c50e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:20.537529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:20.537529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.545809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9adc8b67c9ac441c8dff07d6800fde51', 'request_id': 'req-97106158-bda0-45f8-b4fb-2b97352d9326'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.554147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9adc8b67c9ac441c8dff07d6800fde51', 'request_id': 'req-97106158-bda0-45f8-b4fb-2b97352d9326'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.565797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9adc8b67c9ac441c8dff07d6800fde51', 'request_id': 'req-97106158-bda0-45f8-b4fb-2b97352d9326'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.582474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] The token cache is being invalidate because project 9adc8b67c9ac441c8dff07d6800fde51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:20.582742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.583426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-97106158-bda0-45f8-b4fb-2b97352d9326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.589730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 962/1930] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:20 2022] DELETE /identity/v3/projects/9adc8b67c9ac441c8dff07d6800fde51 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:20.591543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.591871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.592122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.613144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:20.623355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:20.624879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.652519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e68c-7936-4640-95ef-bd9ec2224585 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:20.657118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:20.703670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 963/1931] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:20.704280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:20.710217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.726903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3525a5-ae42-409b-8d6a-025c11e24b0e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:20.732467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:20.733698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.734177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.734620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] PATH_INFO: `/v3/users/e8238738e1b24646a3d3d1272ffb0536` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.734733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:20.735786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.736215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.736450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8238738e1b24646a3d3d1272ffb0536)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:20.736705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] PATH_INFO: `/v3/users/2d14843a17174247bb44fe3caa01e14b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.738279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d14843a17174247bb44fe3caa01e14b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:20.738953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:20.742599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:20.795067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8238738e1b24646a3d3d1272ffb0536'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.800271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d14843a17174247bb44fe3caa01e14b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.806240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8238738e1b24646a3d3d1272ffb0536'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.807693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8238738e1b24646a3d3d1272ffb0536'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.812250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d14843a17174247bb44fe3caa01e14b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.814882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d14843a17174247bb44fe3caa01e14b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.814882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d14843a17174247bb44fe3caa01e14b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.821505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8238738e1b24646a3d3d1272ffb0536'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.830505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7493ef3f-16c6-42df-b640-3e85252d93f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d14843a17174247bb44fe3caa01e14b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.841927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1fc99998-7147-4048-8c55-3b82b77d55af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8238738e1b24646a3d3d1272ffb0536'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:20.850988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 964/1932] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:20 2022] DELETE /identity/v3/users/2d14843a17174247bb44fe3caa01e14b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:20.850988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.850988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.850988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.857043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 969/1933] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:20 2022] DELETE /identity/v3/users/e8238738e1b24646a3d3d1272ffb0536 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:20.858985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:20.859303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:20.859574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:20.883176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:20.884458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.895001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:20.896771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.973195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 970/1934] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:20.992212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:20.992830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 965/1935] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:20.995110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:20.998768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.015356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d55c8-3381-4b2d-987b-389bcab8c5d5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.017710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:21.032746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.033607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.033900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.034186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.034983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:21.035299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.040418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.066423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 966/1936] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:21.066423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.084067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d274f0-1aae-43f3-b717-eff6c53d3180 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.092061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.093257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.093529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a843-30ae-4f3e-958f-c609a9941c50 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.093964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.094442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.097064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.098429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.099096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.099378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.099649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.102433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.118806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.121228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.134245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 971/1937] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:21.154715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 967/1938] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:21.158776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.172023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb92b80-079e-435c-970c-649d8fd8c293 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.240987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.241814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.242108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.242388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.243128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.245710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.272689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 972/1939] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:21.277784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.281569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.287849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.301074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff5348-6ab3-434e-972d-fea4544989a0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.302135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66819282-ad38-4e9c-ab9e-185c17d584cd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.307108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.308392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.308712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.308998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] PATH_INFO: `/v3/projects/72294d2b96c44b37877b0804980578c7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.310364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72294d2b96c44b37877b0804980578c7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.312384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.320383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.323702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.324028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.324314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.325825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.327714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.329883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:21.347090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 968/1940] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:21.348103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.361569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83011d27-9916-4a6d-8e10-72fad0d270a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.368016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.368815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.369746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.369746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.370453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.372352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.385841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 969/1941] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:21.390174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.390772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72294d2b96c44b37877b0804980578c7', 'request_id': 'req-dbaa8b15-83de-4622-8bff-a76659943e23'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.398974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72294d2b96c44b37877b0804980578c7', 'request_id': 'req-dbaa8b15-83de-4622-8bff-a76659943e23'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.407929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2936a-e7ba-4a31-9b75-b32ea82f862d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.409947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72294d2b96c44b37877b0804980578c7', 'request_id': 'req-dbaa8b15-83de-4622-8bff-a76659943e23'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.431225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] The token cache is being invalidate because project 72294d2b96c44b37877b0804980578c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:21.431579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.433310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dbaa8b15-83de-4622-8bff-a76659943e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.439348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 973/1942] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:21 2022] DELETE /identity/v3/projects/72294d2b96c44b37877b0804980578c7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:21.441827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4ec3-346a-4204-805b-0554bb20596f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.442125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4ec3-346a-4204-805b-0554bb20596f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.442410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4ec3-346a-4204-805b-0554bb20596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.465671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d6c4ec3-346a-4204-805b-0554bb20596f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:21.467000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d6c4ec3-346a-4204-805b-0554bb20596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.507050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:21.526370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.527330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.527612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.527910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] PATH_INFO: `/v3/projects/5f609cac9ad54b1a90163e20b58eaad7/users/e7c5ca58e4f847139efe95c56799f0ac/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.578791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f609cac9ad54b1a90163e20b58eaad7, user_id=e7c5ca58e4f847139efe95c56799f0ac, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.582149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753baad-bb89-44bc-8514-bcda9a05faa3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.607784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 974/1943] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:21.612428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 970/1944] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:21 2022] PUT /identity/v3/projects/5f609cac9ad54b1a90163e20b58eaad7/users/e7c5ca58e4f847139efe95c56799f0ac/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:21.618459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.619622 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.620275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.624182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.624484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.624798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.654253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:21.658394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.700823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] MFA Rules not processed for user `e7c5ca58e4f847139efe95c56799f0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:21.702536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.813700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 975/1945] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:21.814662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.816008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.833145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad0496-7a11-41c8-b369-93e194fb6201 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.840878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.841701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.842196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.842596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] PATH_INFO: `/v3/projects/e7b034ee85ae425dbdad0139dfb1901d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.844402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b034ee85ae425dbdad0139dfb1901d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.846909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.865264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 971/1946] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:21.866237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.887600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec13c8-8448-4b10-8e18-48b1be242b0e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.900401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:21.901736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:21.902033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:21.902311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:21.905911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:21.906233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:21.914673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:21.923502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b034ee85ae425dbdad0139dfb1901d', 'request_id': 'req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.931421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b034ee85ae425dbdad0139dfb1901d', 'request_id': 'req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.938377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 972/1947] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:21.943153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.944244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b034ee85ae425dbdad0139dfb1901d', 'request_id': 'req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.946282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:21.962083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f721-4a7b-4a2d-959b-b535357b96d6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:21.970172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] The token cache is being invalidate because project e7b034ee85ae425dbdad0139dfb1901d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:21.970468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.974164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:21.980657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 976/1948] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:21 2022] DELETE /identity/v3/projects/e7b034ee85ae425dbdad0139dfb1901d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:21.982116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:21.984485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.075124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:22.116908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:22.134375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.135117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.135477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.136761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.137720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.140597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.157019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576b5a-0e54-4c02-8b32-2a64c66a64aa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.171539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.172231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.172523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.172817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.173528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.175879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.201705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 973/1949] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:22.203213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.207946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.240592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 977/1950] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:21 2022] POST /identity/v3/projects => generated 383 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:22.241723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.246781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.350815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44355ef5-231f-4b5b-8f98-65c73417c164 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.390738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7328-624a-414b-8a94-fdfa0e6c95df admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.404579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.405618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.409330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.409330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.409744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.411838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.442587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 978/1951] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:22 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:22.443753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.445692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.448135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.448640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.449066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] PATH_INFO: `/v3/users/86f5a715d840469289a65d4b2dfde272` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.450834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f5a715d840469289a65d4b2dfde272)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.453813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.460254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b14ce-28e3-48e0-bde3-a079af4381a1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.466992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.468040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.468547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.469233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.470136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.472883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.506847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 979/1952] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:22 2022] POST /identity/v3/users => generated 292 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:22.517777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.540755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38652321-ae54-4128-9a79-8ef9e0f7316c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.543500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f5a715d840469289a65d4b2dfde272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:22.547779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.548938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.549427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.549812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.550739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.552718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.552939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f5a715d840469289a65d4b2dfde272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:22.555282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f5a715d840469289a65d4b2dfde272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:22.558401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f5a715d840469289a65d4b2dfde272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:22.565684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 980/1953] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:22.567118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.569233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.582019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f5a715d840469289a65d4b2dfde272'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:22.586058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e248cb-3b59-4e0f-b2a9-123eb1e5b2a5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.587255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:22.593925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.595188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.595542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.595910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.597125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.600350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.613242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 974/1954] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:22 2022] DELETE /identity/v3/users/86f5a715d840469289a65d4b2dfde272 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:22.614161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.619082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.624088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 981/1955] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:22.625582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.637323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877322-689d-4ac5-9649-26564d3624a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.723943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.724635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.724940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.725218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.726203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.728146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.745451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 982/1956] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:22.749735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.752563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf52a37-3ad3-4714-97e2-f088692da183 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.758332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.759390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.760178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.760632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] PATH_INFO: `/v3/projects/45db7a027045417792ad0fb909f67e8d/users/28f4a65fcbc147b781e13896c7f71e91/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.775752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6939fa8-c08f-48c4-8bdd-a2dd1967c91a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.780419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45db7a027045417792ad0fb909f67e8d, user_id=28f4a65fcbc147b781e13896c7f71e91, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.783517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.784085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.785365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.785829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.786251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] PATH_INFO: `/v3/projects/2950f7fdce854175b898675361e5c6ed/users/8a754e9e71424c0aa514dd1d28b75fe7/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.816923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 975/1957] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:22 2022] PUT /identity/v3/projects/45db7a027045417792ad0fb909f67e8d/users/28f4a65fcbc147b781e13896c7f71e91/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:22.822093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:22.823649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2950f7fdce854175b898675361e5c6ed, user_id=8a754e9e71424c0aa514dd1d28b75fe7, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:22.826867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ecf51-01b2-4b90-8bbd-36127db643d4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:22.831488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:22.853292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 983/1958] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:22 2022] PUT /identity/v3/projects/2950f7fdce854175b898675361e5c6ed/users/8a754e9e71424c0aa514dd1d28b75fe7/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:22.863853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.864205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.864651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.895804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] MFA Rules not processed for user `8a754e9e71424c0aa514dd1d28b75fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:22.897319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:22.911548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc98e-5690-4025-8bee-148314bdb305 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:22.996772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 984/1959] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:22.997290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:22.998026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:22.998483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:22.998769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:22.999511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.000845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.001276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.002415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.022006 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 976/1960] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:23.024479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.030919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421483c-456a-408f-b73b-488c34ecdadd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.036187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.037277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.037523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1f49-4d15-4dee-b11c-a964b39d33aa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.037638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.039083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] PATH_INFO: `/v3/projects/846dc5fbc6cb4269b80e3147a3584660` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.040511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846dc5fbc6cb4269b80e3147a3584660)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.043117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.043609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.043798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.044125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.044295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.045036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.047064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.074146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:23.083168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 977/1961] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:23.084148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.108041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846dc5fbc6cb4269b80e3147a3584660', 'request_id': 'req-c0841822-8398-439a-942f-f6dd667918c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.114658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846dc5fbc6cb4269b80e3147a3584660', 'request_id': 'req-c0841822-8398-439a-942f-f6dd667918c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.125835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846dc5fbc6cb4269b80e3147a3584660', 'request_id': 'req-c0841822-8398-439a-942f-f6dd667918c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.144892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] The token cache is being invalidate because project 846dc5fbc6cb4269b80e3147a3584660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:23.145087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.145783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.147507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 985/1962] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:22 2022] DELETE /identity/v3/projects/846dc5fbc6cb4269b80e3147a3584660 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:23.148319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.149803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.170264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3174d-991b-4694-84eb-9271cdcb480a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.255664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.256746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.257033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.257308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] PATH_INFO: `/v3/projects/45db7a027045417792ad0fb909f67e8d/users/28f4a65fcbc147b781e13896c7f71e91/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.259381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45db7a027045417792ad0fb909f67e8d, user_id=28f4a65fcbc147b781e13896c7f71e91, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.263798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e4f05-4fe7-423a-8959-5fbdbb102464 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.268610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0841822-8398-439a-942f-f6dd667918c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.277125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.277849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.278138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.278682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.279575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.281585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 978/1963] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:23 2022] PUT /identity/v3/projects/45db7a027045417792ad0fb909f67e8d/users/28f4a65fcbc147b781e13896c7f71e91/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:23.281716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bff992-064c-40a5-98aa-e04664a4df7d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.286526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.287254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.287856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.294790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:23.314927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 986/1964] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:23.317550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098338-b98b-47db-bcdd-683537566b6a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.317797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098338-b98b-47db-bcdd-683537566b6a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.318085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098338-b98b-47db-bcdd-683537566b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.321557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:23.323849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.355332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-37098338-b98b-47db-bcdd-683537566b6a None None] MFA Rules not processed for user `28f4a65fcbc147b781e13896c7f71e91`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:23.356926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-37098338-b98b-47db-bcdd-683537566b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.391632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 979/1965] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:23.392557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.407696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995a8ff-d54d-4f37-8da6-e95b1216b6ce None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.414360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.415251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.415729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.416744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.418288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.424157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1519b-8690-43ff-bc6b-30297b2d1de2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.455991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 980/1966] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:23.457278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.457571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.457858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.468756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 987/1967] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:23.474521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37098338-b98b-47db-bcdd-683537566b6a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.491159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37098338-b98b-47db-bcdd-683537566b6a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.495030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:23.496257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.506507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.507280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 981/1968] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:23.507398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.507840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.508298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.508746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.509287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.511079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.511899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.528249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 988/1969] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:23.529140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.555019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea0e6-e44d-44df-aeb5-a1a8e8398e09 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.564003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:23.574733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.575895 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.576491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.577035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.577978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:23.578482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.580520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.596044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 989/1970] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:23.597493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.610772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce3dff-ae41-4890-9b35-4a135230f14f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.611010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76914-a90e-4736-a816-ea7df3b94ebd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.616909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.617357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.617762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.618122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.618472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.619321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.620095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.620565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] PATH_INFO: `/v3/projects/57cb9c96406c4c1ebb9ed9073455daf5/users/6e6f47a7a7c54d729fa12f846445af53/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.632174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.635550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.646147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cb9c96406c4c1ebb9ed9073455daf5, user_id=6e6f47a7a7c54d729fa12f846445af53, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.647865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 982/1971] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:23.648836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.651911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.665238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.675093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0622-a500-4b9c-8965-d76cda9dd9e2 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.682752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.683744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.684036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.684348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] PATH_INFO: `/v3/users/81bb259fad20492181075f251be475a6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.685891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=81bb259fad20492181075f251be475a6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.688063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.719105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 990/1972] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:23 2022] PUT /identity/v3/projects/57cb9c96406c4c1ebb9ed9073455daf5/users/6e6f47a7a7c54d729fa12f846445af53/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:23.719105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.755793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81bb259fad20492181075f251be475a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.774208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81bb259fad20492181075f251be475a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.774973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81bb259fad20492181075f251be475a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.776523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81bb259fad20492181075f251be475a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.805077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-85352fa3-804a-40ef-a737-04faf7461a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81bb259fad20492181075f251be475a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:23.828500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 983/1973] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:23 2022] DELETE /identity/v3/users/81bb259fad20492181075f251be475a6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:23.832973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.833260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.833560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.835129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70caca63-225f-49ff-a177-6050289203e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:23.864736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] MFA Rules not processed for user `6e6f47a7a7c54d729fa12f846445af53`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:23.865874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.881674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:23.903277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:23.904331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.904898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.905277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.905971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:23.907902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc8ec0-17d7-466e-9101-ba61b5693907 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:23.932999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 984/1974] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:23.933908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:23.935632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.938048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 991/1975] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:23.939346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:23.939635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:23.939914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:23.957171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:23.958357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:23.970765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:24.003525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df005e86-4fd2-4337-a05f-8ed886976b8d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.021842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 992/1976] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:24.022744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.024017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.037363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d007312-f222-40e4-ac42-fb146a5d8c4d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.042650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.043311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.043606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.043886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.044582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.046524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.062534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.063301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.063547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.063913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.065873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.074121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 993/1977] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:24.074997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.076671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.138234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.139975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.150797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 985/1978] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:24.151627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.153098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.167447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d623-0b3e-454a-87fd-aa3cd23adc22 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.174945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.175858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.176332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.176812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.177970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.181350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.218668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 994/1979] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:24.218668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.220566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed24d8-2253-45d6-9516-bd6403dce1b7 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.234500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b187611-8326-4bdd-a4c5-a495ca3ba396 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.242403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.243173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.243879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.245275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.247263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.249946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.275433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 995/1980] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/users => generated 299 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:24.276524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.282942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.284000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.284453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.284960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.295806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab40fd-1ee6-4b6b-889d-35976a9cca70 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.299254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.300803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.303411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.304525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.304994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:24.305166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.305717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.307784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.310646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.313294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 986/1981] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:24.317535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.330346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 996/1982] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:24.334521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.335732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc321fc4-93c5-4f52-a54e-636c5948ab3e service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.341991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.343459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.343762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.344300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.345164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.346033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:24.347474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.358716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230200-4168-46df-b58c-2644cb122305 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.364854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 987/1983] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:24.368853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.374728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.376743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.377267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.377739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.379934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.383703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.388867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ee85d-f485-4c44-8c3b-9a134b9ff378 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.397896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.399962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.400473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.400987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] PATH_INFO: `/v3/projects/88e513aaca0846f3be39e7472670938f/users/e9de8ba6348342bbbff47c6c8a66bb38/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.411099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 997/1984] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:24.415062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.428005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346ab98-ff71-4507-b617-a154e60f367b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.428741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e513aaca0846f3be39e7472670938f, user_id=e9de8ba6348342bbbff47c6c8a66bb38, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.433440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.433809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.434774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.435168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.435543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] PATH_INFO: `/v3/projects/2c695117e91043caa38b75522a67e1f1/users/1818f77cfa3d41fcb4a382843cc432c2/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.456479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c695117e91043caa38b75522a67e1f1, user_id=1818f77cfa3d41fcb4a382843cc432c2, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.458549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 988/1985] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:24 2022] PUT /identity/v3/projects/88e513aaca0846f3be39e7472670938f/users/e9de8ba6348342bbbff47c6c8a66bb38/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:24.462582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810dec8-7cce-43e7-8992-449ec4bd803c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.465080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.527197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 998/1986] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:24 2022] PUT /identity/v3/projects/2c695117e91043caa38b75522a67e1f1/users/1818f77cfa3d41fcb4a382843cc432c2/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:24.539223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.539522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.539804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.557697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aaab8-bd2a-4d6a-9a47-7f12f0566e0a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.591025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] MFA Rules not processed for user `1818f77cfa3d41fcb4a382843cc432c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:24.592420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.645378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.646065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.646646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.646646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.647385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.649086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.664123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 989/1987] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:24.665156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.677751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a5581-9a64-4557-abf3-6f2f2db707f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.683637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.684691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.685078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.685561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] PATH_INFO: `/v3/projects/88e513aaca0846f3be39e7472670938f/users/e9de8ba6348342bbbff47c6c8a66bb38/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.687827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e513aaca0846f3be39e7472670938f, user_id=e9de8ba6348342bbbff47c6c8a66bb38, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.690968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9807be-62de-42c1-babd-1a5a59dda8f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.703916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 990/1988] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:24 2022] PUT /identity/v3/projects/88e513aaca0846f3be39e7472670938f/users/e9de8ba6348342bbbff47c6c8a66bb38/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:24.716248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.716567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.716902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.740718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 999/1989] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:24.746008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.753175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] MFA Rules not processed for user `e9de8ba6348342bbbff47c6c8a66bb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:24.754433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.758500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda4de0-0fcc-4a4d-b503-90d8b6e7575b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.810921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:24.864850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 991/1990] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:24.865781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.869416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.874441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.875120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.875411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.875701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.876389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.878360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.887049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c826cf9-2a60-42f2-8be4-713e2e89f262 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.892228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:24.893135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:24.893423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:24.893831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] PATH_INFO: `/v3/projects/d8c3423c15b34db3b5f8f15cfb0d11a1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:24.894959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c3423c15b34db3b5f8f15cfb0d11a1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:24.897074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:24.908316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1000/1991] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:24.909199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.910923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:24.963503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:24.965228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c3423c15b34db3b5f8f15cfb0d11a1', 'request_id': 'req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:24.972669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c3423c15b34db3b5f8f15cfb0d11a1', 'request_id': 'req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:24.975031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6bcfa-a8bb-4e94-ae75-520f916d55e3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:24.980919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c3423c15b34db3b5f8f15cfb0d11a1', 'request_id': 'req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:24.991502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] The token cache is being invalidate because project d8c3423c15b34db3b5f8f15cfb0d11a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:24.991794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:24.992460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:24.994400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 992/1992] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:24 2022] DELETE /identity/v3/projects/d8c3423c15b34db3b5f8f15cfb0d11a1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:24.995345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:24.996709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.033913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.034554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.034846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.035130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.037028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.063897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815bcc-0cb8-4a73-b326-d59f8ee84a2a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.068734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.069322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.069603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.069889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.070581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.073287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.093894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 993/1993] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:24 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:25.093894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.093894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.099338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.101070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.108961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833542e8-5ed1-4092-9f76-786aa7e202ac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.111250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1001/1994] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:25.112045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.114560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.115428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.115712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.115991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] PATH_INFO: `/v3/users/cb1e9a5138c64c2f83bd47718b4da99b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.117246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1e9a5138c64c2f83bd47718b4da99b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.119271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.122705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b438-a554-4587-890f-b4792d22dde1 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.128794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.129460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.129749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.130029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.130762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.133122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.148543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1002/1995] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:25.153151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.166457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591995-7d8b-4c8e-9c6f-daad9f27fa0d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.171667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.172643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.173018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.173382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] PATH_INFO: `/v3/projects/0fe83e746e25483b9fa6745db7c56e54/users/5c20e39c2d2248b99e645436e1d5dd2d/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.180494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:25.183256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1e9a5138c64c2f83bd47718b4da99b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.189882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe83e746e25483b9fa6745db7c56e54, user_id=5c20e39c2d2248b99e645436e1d5dd2d, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.192502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.201220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1e9a5138c64c2f83bd47718b4da99b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.201974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1e9a5138c64c2f83bd47718b4da99b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.202856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1e9a5138c64c2f83bd47718b4da99b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.211349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1003/1996] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:25 2022] PUT /identity/v3/projects/0fe83e746e25483b9fa6745db7c56e54/users/5c20e39c2d2248b99e645436e1d5dd2d/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:25.216646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.219097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1e9a5138c64c2f83bd47718b4da99b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.219592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.235499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 994/1997] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:25 2022] DELETE /identity/v3/users/cb1e9a5138c64c2f83bd47718b4da99b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:25.298561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9f19-84f8-4dd6-9ca9-48018c1e82c1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.329161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:25.369022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.370544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.370882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.370882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.371663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.373839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a909ef4-98b8-4462-898b-0a91c146a179 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.386950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1004/1998] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:25.391457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.403642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94705e-0be9-46f9-8486-f0415857f8d0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.409989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.410770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.411195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.411504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] PATH_INFO: `/v3/projects/0fe83e746e25483b9fa6745db7c56e54/users/5c20e39c2d2248b99e645436e1d5dd2d/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.414004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe83e746e25483b9fa6745db7c56e54, user_id=5c20e39c2d2248b99e645436e1d5dd2d, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.417426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.430719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 995/1999] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:25 2022] PUT /identity/v3/projects/0fe83e746e25483b9fa6745db7c56e54/users/5c20e39c2d2248b99e645436e1d5dd2d/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:25.443109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.443525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.443959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.480733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] MFA Rules not processed for user `5c20e39c2d2248b99e645436e1d5dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:25.484306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.554716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.563600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.587566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1005/2000] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:25.602439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.604462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.644767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591fc21-d26b-4a57-8f3c-6db92d91c999 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.649523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.650934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.651270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.651645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] PATH_INFO: `/v3/projects/a63d7fc04dc94028b55362bdeece48f7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.653462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63d7fc04dc94028b55362bdeece48f7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.654774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.670101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ff676-cf0c-4bc6-b7e8-5d9177775f5a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.675091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.676193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.676193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.676595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.678583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.706667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63d7fc04dc94028b55362bdeece48f7', 'request_id': 'req-917876a2-a87e-45c1-95a2-46a8dd8a09a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.713233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63d7fc04dc94028b55362bdeece48f7', 'request_id': 'req-917876a2-a87e-45c1-95a2-46a8dd8a09a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.721786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63d7fc04dc94028b55362bdeece48f7', 'request_id': 'req-917876a2-a87e-45c1-95a2-46a8dd8a09a5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.735254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] The token cache is being invalidate because project a63d7fc04dc94028b55362bdeece48f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:25.735558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.736262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-917876a2-a87e-45c1-95a2-46a8dd8a09a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:25.738504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 996/2001] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:25 2022] DELETE /identity/v3/projects/a63d7fc04dc94028b55362bdeece48f7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:25.742992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.743291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.743577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.758517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.760004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413cc97-a91b-4b6d-bc4e-3f73bef12fc4 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.762102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:25.763310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.769465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1006/2002] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:25.797410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:25.829318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 997/2003] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:25.837758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.838242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.838543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.845175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:25.859395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:25.860650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:25.871572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1007/2004] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:25.875884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.885740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cfc49-8f71-4787-96af-e0c10ff3dc43 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.891153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.892052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.892572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.892964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.893896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:25.894306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.896132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.912665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 998/2005] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:25.914488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.924933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8ef1f-0fd5-4e3a-9bda-d826ae8a0ee7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:25.929921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:25.930636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:25.930919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:25.931193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:25.932679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:25.935453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:25.967100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1008/2006] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:25 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:25.968180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:25.980145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1786a-411a-46d6-88e1-058123546deb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.063402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.063402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.063402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.063402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.063938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.063938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.085421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 999/2007] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:25 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:26.090032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.102287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a01eb-03a9-4b52-9759-4633d795bfbb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.112561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.113987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.114289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.114579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.115343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.117204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.135811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1009/2008] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:26.140048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.151727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ff3d8-4843-4b3f-b1e8-e295d0df7d4f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.157590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.158684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.158976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.159397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.160144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.161807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275cec4-a12c-4b5d-a6c9-ad48e355e111 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.176655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1000/2009] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:26.178089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.189005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fad1f-bf29-4502-b9e5-fb08211eb8ac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.193509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.194380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.194659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.194932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] PATH_INFO: `/v3/projects/1195e940d95149829277706820f7c6b5/users/31f6e3496849442dbf88b5020ef35946/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.210338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1195e940d95149829277706820f7c6b5, user_id=31f6e3496849442dbf88b5020ef35946, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.213102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.226127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1010/2010] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:26 2022] PUT /identity/v3/projects/1195e940d95149829277706820f7c6b5/users/31f6e3496849442dbf88b5020ef35946/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:26.237409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.237710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.237995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.265623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] MFA Rules not processed for user `31f6e3496849442dbf88b5020ef35946`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:26.266819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:26.329743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1001/2011] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:26.334285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.396196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2cce1-038c-4e37-9b38-b14b9bed8eb2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.400951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.401603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.401888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.402165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.402857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.404821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.429502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1011/2012] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:26 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:26.434157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.444994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766b3de-2150-4dda-a7c3-aa1f51fe47d1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.479104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:26.506123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.506796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.507352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.507352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.508032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.509988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.530219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1002/2013] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:26 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:26.534705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.546864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec165dd-ccea-4a4c-add2-04cb21404332 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.551951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.552640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.552928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.553202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.553926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.555552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.567503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1012/2014] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:26.571603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.581536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b6baa-c309-41f5-946b-025e73f25d9e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.586063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.586924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.587202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.587473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] PATH_INFO: `/v3/projects/0367b050b2de434da6165a5300f6bb25/users/623e4dcdfee54161afeda7d67dc7fefd/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.603033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0367b050b2de434da6165a5300f6bb25, user_id=623e4dcdfee54161afeda7d67dc7fefd, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.606930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.621271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1003/2015] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:26 2022] PUT /identity/v3/projects/0367b050b2de434da6165a5300f6bb25/users/623e4dcdfee54161afeda7d67dc7fefd/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:26.625301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.635069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17073f3e-6b69-45f1-8d2f-7e7b262728dd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.690521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.691184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.691463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.691739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.692455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.694058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8b3d3-848c-4ce8-86a3-78a9a70866d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.703627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1013/2016] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:26.707956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.718152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11c4d3-8d43-464a-9b83-8f566b4d243d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.722747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.723629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.723917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.724199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] PATH_INFO: `/v3/projects/0367b050b2de434da6165a5300f6bb25/users/623e4dcdfee54161afeda7d67dc7fefd/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.726475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0367b050b2de434da6165a5300f6bb25, user_id=623e4dcdfee54161afeda7d67dc7fefd, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.729738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.739933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1004/2017] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:26 2022] PUT /identity/v3/projects/0367b050b2de434da6165a5300f6bb25/users/623e4dcdfee54161afeda7d67dc7fefd/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:26.750748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.751074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.751364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.770839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:26.781747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] MFA Rules not processed for user `623e4dcdfee54161afeda7d67dc7fefd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:26.782949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:26.858996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1014/2018] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:26.870525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:26.872966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:26.934027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c3947-f2b4-4b0d-92a6-2f55daf2a143 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:26.937977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:26.938630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:26.938912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:26.939189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:26.950893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:26.952401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2bfd-5f55-451e-98be-c1d91d09a9f6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:26.964988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1005/2019] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:28.314299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:28.380257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef4c74-0334-42bb-a848-9c1360b6f525 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:28.385022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:28.385892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:28.386179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:28.386453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] PATH_INFO: `/v3/users/31f6e3496849442dbf88b5020ef35946` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:28.387668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f6e3496849442dbf88b5020ef35946)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:28.389636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:28.436081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f6e3496849442dbf88b5020ef35946'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:28.443731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f6e3496849442dbf88b5020ef35946'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:28.444405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f6e3496849442dbf88b5020ef35946'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:28.452205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f6e3496849442dbf88b5020ef35946'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:28.465266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f6e3496849442dbf88b5020ef35946'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:28.471382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1015/2020] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:28 2022] DELETE /identity/v3/users/31f6e3496849442dbf88b5020ef35946 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:28.476035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:28.476521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:28.476610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:28.495332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:28.496576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:28.559117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1006/2021] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:28.568994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:28.570409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:28.585149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:28.631636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea137f-641c-47bd-93d9-de6d58c4cabf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:28.635494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:28.636150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:28.636438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:28.636746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:28.647851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:28.649489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:28.661118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1016/2022] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:28.939127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:28.941296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:28.960097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5af9a-ef3d-4ed9-8a9b-2b7f32ce79be None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:28.967271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:28.968151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:28.968434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:28.968753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] PATH_INFO: `/v3/projects/1195e940d95149829277706820f7c6b5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:28.970833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1195e940d95149829277706820f7c6b5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:28.972527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:29.006997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:29.028724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1195e940d95149829277706820f7c6b5', 'request_id': 'req-7799c321-d7ca-4522-96a4-7af2e94da1db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.035186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1195e940d95149829277706820f7c6b5', 'request_id': 'req-7799c321-d7ca-4522-96a4-7af2e94da1db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.044701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1195e940d95149829277706820f7c6b5', 'request_id': 'req-7799c321-d7ca-4522-96a4-7af2e94da1db'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.055840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] The token cache is being invalidate because project 1195e940d95149829277706820f7c6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:29.056050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.057467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.059172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1007/2023] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:28 2022] DELETE /identity/v3/projects/1195e940d95149829277706820f7c6b5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:29.063269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:29.074137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69dedb-2230-4eaf-82d0-743d321e971d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:29.139133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:29.140023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:29.140301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:29.140585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] PATH_INFO: `/v3/users/623e4dcdfee54161afeda7d67dc7fefd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:29.141819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=623e4dcdfee54161afeda7d67dc7fefd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:29.143765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:29.206918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623e4dcdfee54161afeda7d67dc7fefd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.215660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623e4dcdfee54161afeda7d67dc7fefd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.216356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623e4dcdfee54161afeda7d67dc7fefd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.223593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623e4dcdfee54161afeda7d67dc7fefd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.239243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b42f1-b135-429b-adee-40102fd933f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623e4dcdfee54161afeda7d67dc7fefd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.246264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1017/2024] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:29 2022] DELETE /identity/v3/users/623e4dcdfee54161afeda7d67dc7fefd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:29.574190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:29.575636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:29.610975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799c321-d7ca-4522-96a4-7af2e94da1db admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:29.698671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:29.699909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:29.700167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:29.700693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] PATH_INFO: `/v3/projects/0367b050b2de434da6165a5300f6bb25` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:29.702051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0367b050b2de434da6165a5300f6bb25)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:29.704086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:29.826171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0367b050b2de434da6165a5300f6bb25', 'request_id': 'req-e36e73f6-d80e-4c87-8d10-0f7804221068'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.850792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0367b050b2de434da6165a5300f6bb25', 'request_id': 'req-e36e73f6-d80e-4c87-8d10-0f7804221068'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.864004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0367b050b2de434da6165a5300f6bb25', 'request_id': 'req-e36e73f6-d80e-4c87-8d10-0f7804221068'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.890592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] The token cache is being invalidate because project 0367b050b2de434da6165a5300f6bb25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:29.891047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.893451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:29.898494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1008/2025] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:29 2022] DELETE /identity/v3/projects/0367b050b2de434da6165a5300f6bb25 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:29.904693 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863485d-98c8-4904-a79f-0300971a5ece None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:29.905161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863485d-98c8-4904-a79f-0300971a5ece None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:29.905323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863485d-98c8-4904-a79f-0300971a5ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:29.929803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0863485d-98c8-4904-a79f-0300971a5ece None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:29.931050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0863485d-98c8-4904-a79f-0300971a5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:29.978872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:29.983742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:29.986004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:30.034314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1018/2026] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:30.042594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.042892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.043179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.049820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:30.065337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e73f6-d80e-4c87-8d10-0f7804221068 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.067423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:30.068997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:30.069873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.070604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.070983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.071396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.074365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:30.083234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1019/2027] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:30.087091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.096535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7455e5-4dc7-47f4-b4e9-8f71792721cf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.101935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.102678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.103032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.103506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.104901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:30.105505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.108568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.126448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1020/2028] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:02:30.127827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.139612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a71f20-386b-41a0-b937-a28ad57a37e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.147268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.148032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.148352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.148771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.149441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.149550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.151064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.151431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.164932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1009/2029] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:30.183602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1021/2030] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:30.187763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.230495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d96dc3-ded1-4160-9846-d00edfa7a052 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.348991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.350316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.350987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.351484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.352458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.355606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.386699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1010/2031] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:30.388004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.404073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5f09-61ce-467e-bf14-f3a512253151 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.410552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.411274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.411561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.411909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.412765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.414544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.430717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1022/2032] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:30.435087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.446551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567541-87c8-4c1d-9207-bc5959f7798f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.452320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.453326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.453617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.453913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.454705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.456330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ac572-679f-43a0-9735-32ae1fda6cf8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.470114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1011/2033] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:30.474780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.487255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbaa9-94aa-48be-9127-2185d8741a06 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.492113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.493167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.493673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.494168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] PATH_INFO: `/v3/projects/2b3b131898f141b6bb134b7794f1d76f/users/4620148f5a434887ad37de382f27a164/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.510863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3b131898f141b6bb134b7794f1d76f, user_id=4620148f5a434887ad37de382f27a164, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.514081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.530446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1023/2034] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:30 2022] PUT /identity/v3/projects/2b3b131898f141b6bb134b7794f1d76f/users/4620148f5a434887ad37de382f27a164/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:30.540925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.544460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.544912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.575328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] MFA Rules not processed for user `4620148f5a434887ad37de382f27a164`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:30.576672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:30.683168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1012/2035] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:30.688063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.790354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbaa8b-f1b0-46d8-9dcc-e1dd92c9c3f4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.795808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.796639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.797019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.797372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.798352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.802421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:30.862853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1024/2036] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:30.864608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:30.877519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81c8d-3dff-40e4-9685-48ac9c48264b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:30.881663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:30.969005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:30.970377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:30.970765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:30.971144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:30.976742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:30.979024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.004796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1013/2037] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:30 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:31.009552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.022067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9be04-cd18-495d-9c76-0a8317e25d18 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.028409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.029255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.029643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.030207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.031046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.033960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.052548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1025/2038] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:31.053963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.061148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.066561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3999785-cb74-475b-ac09-eee320edab02 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.066947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.071497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.072806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.072990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.073332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] PATH_INFO: `/v3/projects/569285494aa647e2a8aada53976a8fba/users/f758d6973d634b6c9cb5d2506a2509e0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.092691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=569285494aa647e2a8aada53976a8fba, user_id=f758d6973d634b6c9cb5d2506a2509e0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.095336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.111068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1014/2039] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:31 2022] PUT /identity/v3/projects/569285494aa647e2a8aada53976a8fba/users/f758d6973d634b6c9cb5d2506a2509e0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:31.114958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:31.116000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.148837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc33417-755e-4e1c-b7c1-db197d7c91ce admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.200796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980c548-f036-49d5-85e9-3c01fd80d61a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.205486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.206163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.206452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.206748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.208557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.209301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.221320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.222008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1015/2040] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:31.225029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.226318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.226654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.226801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.230309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.238737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ef75c-6d6b-4bff-a668-52b39a553912 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.245619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.246512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.246801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.247120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] PATH_INFO: `/v3/projects/569285494aa647e2a8aada53976a8fba/users/f758d6973d634b6c9cb5d2506a2509e0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.249696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569285494aa647e2a8aada53976a8fba, user_id=f758d6973d634b6c9cb5d2506a2509e0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.252113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f8718-dfcd-4e08-a0d5-ac6b535a9776 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.264214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1016/2041] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:31 2022] PUT /identity/v3/projects/569285494aa647e2a8aada53976a8fba/users/f758d6973d634b6c9cb5d2506a2509e0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:31.274400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.274697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.274982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.310163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] MFA Rules not processed for user `f758d6973d634b6c9cb5d2506a2509e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:31.311381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.319667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.321436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.332856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1026/2042] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:31.368244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:31.399916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1017/2043] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:31.413136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.423368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cfa2-7ee4-4e3d-8d91-748e07508d8f service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.483574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.484229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.484520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.484836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.496269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.497833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb67ac-c203-4ff3-81df-77128720c604 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.508750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1027/2044] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:31.582121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.650635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58e3d1-067a-49e6-80b1-51da8e48a6c9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.655795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.656775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.657061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.657380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] PATH_INFO: `/v3/users/4620148f5a434887ad37de382f27a164` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.658621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4620148f5a434887ad37de382f27a164)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.660559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.721611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4620148f5a434887ad37de382f27a164'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:31.730080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4620148f5a434887ad37de382f27a164'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:31.730803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4620148f5a434887ad37de382f27a164'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:31.731410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4620148f5a434887ad37de382f27a164'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:31.744956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4620148f5a434887ad37de382f27a164'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:31.758592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1018/2045] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:31 2022] DELETE /identity/v3/users/4620148f5a434887ad37de382f27a164 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:31.764025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.764315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.764671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.782867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:31.784120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.825659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:31.849585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1028/2046] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:31.860379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.861861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.920961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721ebf-3c15-420f-9838-8e3f40ace098 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.924702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.925400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.925648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.926703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:31.927093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.928138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:31.937582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.939044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f393f-e54a-4547-b57c-374ef1fa80fb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.940996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f405-1b0a-48df-bba9-65f8f22990cb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:31.948138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:31.949095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:31.949384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:31.949667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] PATH_INFO: `/v3/users/1818f77cfa3d41fcb4a382843cc432c2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:31.950955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1818f77cfa3d41fcb4a382843cc432c2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:31.952932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:31.953879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1019/2047] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:32.013942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1818f77cfa3d41fcb4a382843cc432c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.023839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1818f77cfa3d41fcb4a382843cc432c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.024502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1818f77cfa3d41fcb4a382843cc432c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.035152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1818f77cfa3d41fcb4a382843cc432c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.052270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1818f77cfa3d41fcb4a382843cc432c2'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.064578 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1029/2048] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:31 2022] DELETE /identity/v3/users/1818f77cfa3d41fcb4a382843cc432c2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:32.066685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.067043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.067267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.079060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:32.097693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:32.099142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.188701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1020/2049] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:32.194371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.195888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.261695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.263189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.269885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306a570-8434-41d0-90ca-5f0989c2be92 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.274823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.276026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14b8ab-7e00-4b80-98a6-2d9f32de3f9e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.276375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.276864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.277243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.283923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.284818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.285101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.285419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] PATH_INFO: `/v3/users/e7c5ca58e4f847139efe95c56799f0ac` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.287429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c5ca58e4f847139efe95c56799f0ac)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.289788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.292389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.294092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.306012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1030/2050] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:32.355864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.357422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.366049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c5ca58e4f847139efe95c56799f0ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.376543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97844-ac8a-4a3c-8ddd-e1e352884ee0 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.379442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c5ca58e4f847139efe95c56799f0ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.380853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c5ca58e4f847139efe95c56799f0ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.382614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.383393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c5ca58e4f847139efe95c56799f0ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.383506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.383809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.384024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] PATH_INFO: `/v3/projects/2b3b131898f141b6bb134b7794f1d76f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.385343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3b131898f141b6bb134b7794f1d76f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.387307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.408303 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-988ae855-9018-4199-94c9-44fbe16b334c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c5ca58e4f847139efe95c56799f0ac'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.436155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1021/2051] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:32 2022] DELETE /identity/v3/users/e7c5ca58e4f847139efe95c56799f0ac => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:32.438026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.438461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.438888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.457643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3b131898f141b6bb134b7794f1d76f', 'request_id': 'req-bcd46221-f56a-4784-8e8b-0936febfa775'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.460980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:32.462889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:32.464292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.467797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3b131898f141b6bb134b7794f1d76f', 'request_id': 'req-bcd46221-f56a-4784-8e8b-0936febfa775'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.483436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3b131898f141b6bb134b7794f1d76f', 'request_id': 'req-bcd46221-f56a-4784-8e8b-0936febfa775'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.500723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] The token cache is being invalidate because project 2b3b131898f141b6bb134b7794f1d76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:32.501731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.502828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.512097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1031/2052] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:32 2022] DELETE /identity/v3/projects/2b3b131898f141b6bb134b7794f1d76f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:32.513528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.526322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.555275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1022/2053] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:32.566874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.577509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.618084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:32.660255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd46221-f56a-4784-8e8b-0936febfa775 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.666186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.667373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.667736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.668149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] PATH_INFO: `/v3/users/f758d6973d634b6c9cb5d2506a2509e0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.670278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f758d6973d634b6c9cb5d2506a2509e0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.672572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.709692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f070923-616c-43b6-9de0-0106cf4dd0d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.715288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.716189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.716481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.716832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.732469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.734075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.760971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f758d6973d634b6c9cb5d2506a2509e0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.762940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1023/2054] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:32.766394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.769449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:32.770401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f758d6973d634b6c9cb5d2506a2509e0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.771344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f758d6973d634b6c9cb5d2506a2509e0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.781472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f758d6973d634b6c9cb5d2506a2509e0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.797459 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329b398-62e4-4a0b-80a1-c527b1ee5998 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:32.798803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f758d6973d634b6c9cb5d2506a2509e0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.804832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:32.805989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:32.806990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:32.807395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] PATH_INFO: `/v3/projects/2c695117e91043caa38b75522a67e1f1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:32.810544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c695117e91043caa38b75522a67e1f1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:32.811633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:32.811899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1032/2055] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:32 2022] DELETE /identity/v3/users/f758d6973d634b6c9cb5d2506a2509e0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:32.877018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c695117e91043caa38b75522a67e1f1', 'request_id': 'req-208fc5d0-1827-4f34-9207-de8326d9e577'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.885950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c695117e91043caa38b75522a67e1f1', 'request_id': 'req-208fc5d0-1827-4f34-9207-de8326d9e577'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.897611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c695117e91043caa38b75522a67e1f1', 'request_id': 'req-208fc5d0-1827-4f34-9207-de8326d9e577'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.916379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] The token cache is being invalidate because project 2c695117e91043caa38b75522a67e1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:32.917131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.917976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:32.920038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1024/2056] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:32 2022] DELETE /identity/v3/projects/2c695117e91043caa38b75522a67e1f1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:32.924274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:32.925978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:33.091690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd7162-f304-4771-9188-47a42a0f068a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:33.099106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:33.224091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:33.225250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.225861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.226223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] PATH_INFO: `/v3/users/5c20e39c2d2248b99e645436e1d5dd2d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.227611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c20e39c2d2248b99e645436e1d5dd2d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:33.230194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:33.326489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c20e39c2d2248b99e645436e1d5dd2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.340183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c20e39c2d2248b99e645436e1d5dd2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.341062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c20e39c2d2248b99e645436e1d5dd2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.350400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c20e39c2d2248b99e645436e1d5dd2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.356442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:33.357969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:33.376876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fc5d0-1827-4f34-9207-de8326d9e577 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:33.385601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:33.387230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.388173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c20e39c2d2248b99e645436e1d5dd2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.388295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.388767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] PATH_INFO: `/v3/projects/5f609cac9ad54b1a90163e20b58eaad7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.392018 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f609cac9ad54b1a90163e20b58eaad7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:33.395139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:33.396736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1033/2057] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:32 2022] DELETE /identity/v3/users/5c20e39c2d2248b99e645436e1d5dd2d => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:33.398291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:33.401634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:33.469703 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f609cac9ad54b1a90163e20b58eaad7', 'request_id': 'req-70838797-3581-43f2-a7b4-437f0a234512'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.481042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f609cac9ad54b1a90163e20b58eaad7', 'request_id': 'req-70838797-3581-43f2-a7b4-437f0a234512'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.492668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f609cac9ad54b1a90163e20b58eaad7', 'request_id': 'req-70838797-3581-43f2-a7b4-437f0a234512'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.506821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb1c1-a781-45fb-b395-2ceab82bf9aa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:33.516916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] The token cache is being invalidate because project 5f609cac9ad54b1a90163e20b58eaad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:33.517207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.517876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.519767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1025/2058] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:33 2022] DELETE /identity/v3/projects/5f609cac9ad54b1a90163e20b58eaad7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:33.523203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:33.526578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:33.552928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:33.584920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:33.601323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:33.604059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.604478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.604898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] PATH_INFO: `/v3/projects/569285494aa647e2a8aada53976a8fba` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.606627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569285494aa647e2a8aada53976a8fba)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:33.609282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:33.635732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70838797-3581-43f2-a7b4-437f0a234512 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:33.641116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:33.642003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.642341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.642627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] PATH_INFO: `/v3/users/28f4a65fcbc147b781e13896c7f71e91` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.643875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f4a65fcbc147b781e13896c7f71e91)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:33.645965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:33.681533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569285494aa647e2a8aada53976a8fba', 'request_id': 'req-a7d166f1-1949-467f-86ea-13bacca5ddb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.693590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569285494aa647e2a8aada53976a8fba', 'request_id': 'req-a7d166f1-1949-467f-86ea-13bacca5ddb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.706394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569285494aa647e2a8aada53976a8fba', 'request_id': 'req-a7d166f1-1949-467f-86ea-13bacca5ddb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.739501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] The token cache is being invalidate because project 569285494aa647e2a8aada53976a8fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:33.739806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.740567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a7d166f1-1949-467f-86ea-13bacca5ddb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.742434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1034/2059] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:33 2022] DELETE /identity/v3/projects/569285494aa647e2a8aada53976a8fba => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:33.748114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.748426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f4a65fcbc147b781e13896c7f71e91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.748604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.749052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.770003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f4a65fcbc147b781e13896c7f71e91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.770797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f4a65fcbc147b781e13896c7f71e91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.771842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f4a65fcbc147b781e13896c7f71e91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.777926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:33.779624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:33.809192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0501ccbf-fb61-4284-8880-724c2d844b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f4a65fcbc147b781e13896c7f71e91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:33.838719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1026/2060] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:33 2022] DELETE /identity/v3/users/28f4a65fcbc147b781e13896c7f71e91 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:33.866754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1035/2061] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:33.870120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:33.870420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:33.870755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:33.896649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:33.897928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.013936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1027/2062] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:34.023950 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.052645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee95dd-87d9-4ac4-ba84-025fa39b773f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.060864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.061583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.061813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.062565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.065061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:34.065061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.066468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.109042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1036/2063] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:34.113643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.131258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69930d5-e4a1-40c1-ac4a-817c83731a7f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.132211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.136965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.145554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.146245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.146531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.146812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.156008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.162212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bccde-eaee-4063-ab9d-21d237403167 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.169939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.200638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.203329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.204837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.205818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] PATH_INFO: `/v3/projects/0fe83e746e25483b9fa6745db7c56e54` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.216518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe83e746e25483b9fa6745db7c56e54)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.222380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.269160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1028/2064] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:34.273823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.314993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:34.356275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:34.361252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe83e746e25483b9fa6745db7c56e54', 'request_id': 'req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:34.381450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe83e746e25483b9fa6745db7c56e54', 'request_id': 'req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:34.385834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d7f6-420a-452b-b117-addd8dad89d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.405838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe83e746e25483b9fa6745db7c56e54', 'request_id': 'req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:34.423352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] The token cache is being invalidate because project 0fe83e746e25483b9fa6745db7c56e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:34.423706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:34.424605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c32caca7-5a95-4c0f-9470-f85a7d9bfca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:34.426990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1037/2065] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:34 2022] DELETE /identity/v3/projects/0fe83e746e25483b9fa6745db7c56e54 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:34.433637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64552-5bc3-4136-86fc-249b3e9f18ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.434180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64552-5bc3-4136-86fc-249b3e9f18ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.434829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64552-5bc3-4136-86fc-249b3e9f18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.455483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bad64552-5bc3-4136-86fc-249b3e9f18ad None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:34.457828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bad64552-5bc3-4136-86fc-249b3e9f18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.493859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.495080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.495761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.496163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.497171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.501198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.541365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1029/2066] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/users => generated 300 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:34.545469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.548712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.579780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b643-aef6-4526-bd3c-bc2d51d8e60c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.580358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1038/2067] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:34.589314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.589740 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.589995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.596229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.597351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.597827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.598516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.600795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.604041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.626558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:34.627073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1030/2068] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:34.627909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.628339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.645711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1039/2069] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:34.646872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96a31a-a6fc-41f3-b722-75d8ab128382 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.665437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.666195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.666195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.666195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.666195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.666516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.666516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.677703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5239dd-7a6a-4d47-9e50-3b1ec3500224 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.677879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1031/2070] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:34.678934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.684317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.685296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.685756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.686369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.687937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:34.688424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.690673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.696113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519b9f-9744-44ed-8bbb-07d35f87e2f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.702324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.708076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1040/2071] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:34.712288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.715289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.717940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.718565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] PATH_INFO: `/v3/projects/f1290ce33e624a0e9f3570a7769f745f/users/60756d1f09274d1cb6d395802c44895a/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.737449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeced51-bcb5-4d0c-aae1-62962efc72b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.750361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.753354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.753354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.754231 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.755130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.758336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.775449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1290ce33e624a0e9f3570a7769f745f, user_id=60756d1f09274d1cb6d395802c44895a, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.779679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a889df-bf2b-42e8-89b1-9af0bd251a3b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.810793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1032/2072] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:34 2022] PUT /identity/v3/projects/f1290ce33e624a0e9f3570a7769f745f/users/60756d1f09274d1cb6d395802c44895a/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:34.818317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.818807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.819068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.833203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1041/2073] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:34.834135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.877630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] MFA Rules not processed for user `60756d1f09274d1cb6d395802c44895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:34.879888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.926888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf52fb-f703-4fb0-8e2b-b64d0746482b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.939877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:34.940754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:34.941162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:34.941604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:34.942507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:34.944838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:34.974955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1042/2074] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:34.976193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.976536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1033/2075] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:34.980244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:34.985435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:34.991142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:34.996673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e347fe-58d6-4a7c-a657-9d15de45559b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:34.999635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43edea-a797-4ba1-bc40-6c51a374923c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.002333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.003255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.003548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.003828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] PATH_INFO: `/v3/projects/45db7a027045417792ad0fb909f67e8d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.005185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45db7a027045417792ad0fb909f67e8d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.005617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.006325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.006614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.006905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.007326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.007632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.009409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.020232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1034/2076] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.021167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.033229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:35.034109 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1115af4-7c77-4634-a734-21bd2beb5e2d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.039380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.040035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.040321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.041158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.041883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.044014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.067410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45db7a027045417792ad0fb909f67e8d', 'request_id': 'req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:35.071114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1035/2077] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:35.072106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.074502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45db7a027045417792ad0fb909f67e8d', 'request_id': 'req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:35.083275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45db7a027045417792ad0fb909f67e8d', 'request_id': 'req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:35.095009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] The token cache is being invalidate because project 45db7a027045417792ad0fb909f67e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:35.095383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:35.096100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:35.097801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1043/2078] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:34 2022] DELETE /identity/v3/projects/45db7a027045417792ad0fb909f67e8d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:35.098592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.099875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.154788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2216b-41d7-43b9-b389-803af970b18a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.159857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.160525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.160860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.161139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.161891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.163495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f36df-0cf9-4ba5-97c3-cce90d806404 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.168886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7865-e3cd-41e4-ac49-7a4a10a55cd8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.173805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.174485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.174592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1036/2079] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.174774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.175054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.175758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.176322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198270-ec1e-4476-883a-fad54801a838 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.176655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198270-ec1e-4476-883a-fad54801a838 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.176928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198270-ec1e-4476-883a-fad54801a838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.177976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.196692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60198270-ec1e-4476-883a-fad54801a838 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:35.198163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60198270-ec1e-4476-883a-fad54801a838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.203633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1044/2080] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:35.203633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.203633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.207224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1037/2081] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:35.207832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60198270-ec1e-4476-883a-fad54801a838 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.218322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60198270-ec1e-4476-883a-fad54801a838 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.219715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac281a-267d-4af5-ab43-ae05d6dbb217 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.222913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.223746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.224028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.224298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.224514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.225152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.225705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.226018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.226298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] PATH_INFO: `/v3/projects/735f29ca5dd9418e92da9b80a9373ac8/users/34bd2ff763fc4e12999884464df92e4e/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.227053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665924-0bf7-4986-a533-b2f70ffb2801 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.238152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1038/2082] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.239648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0eb29-8c9a-4c11-87e7-a6cbcafc9b5b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.239952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0eb29-8c9a-4c11-87e7-a6cbcafc9b5b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.240233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0eb29-8c9a-4c11-87e7-a6cbcafc9b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.246076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] RBAC: Authorizing `identity:create_grant(project_id=735f29ca5dd9418e92da9b80a9373ac8, user_id=34bd2ff763fc4e12999884464df92e4e, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.248933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.259904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-78b0eb29-8c9a-4c11-87e7-a6cbcafc9b5b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:35.261242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78b0eb29-8c9a-4c11-87e7-a6cbcafc9b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.262009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1045/2083] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:35 2022] PUT /identity/v3/projects/735f29ca5dd9418e92da9b80a9373ac8/users/34bd2ff763fc4e12999884464df92e4e/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:35.269363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.332492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1039/2084] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:35.334077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8f513-e183-4a3a-b24b-95296634533e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.336945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.337313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.338187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.340539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.341531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.341816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.342597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] PATH_INFO: `/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2/users/8fa3cc03ae39441caa65770b0c9bd3a1/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.362072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae16d3ac0c33436d93d1d32c4f691ac2, user_id=8fa3cc03ae39441caa65770b0c9bd3a1, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.364882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.367308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:35.368759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] MFA Rules not processed for user `34bd2ff763fc4e12999884464df92e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:35.372228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.387454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1046/2085] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:35 2022] PUT /identity/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2/users/8fa3cc03ae39441caa65770b0c9bd3a1/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:35.388544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.426799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:35.468832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30382d-fbba-4d25-8d2e-2e33bea9b2b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.474414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.475752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.476116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.476480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.477453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:35.478835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.480692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.495270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1047/2086] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:35.496221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.498323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1040/2087] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:35.499998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.517885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235e371-76ce-4535-82b4-eafbb4675264 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.522711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7262-a780-4608-b580-6d4f02172858 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.525911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.526776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.527534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.528303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.529050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.530891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.534510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.534510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.534872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.534872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.535705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.537659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.547496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1048/2088] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.549405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.558538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.581815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1041/2089] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:35.583514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.649752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1365593-b08b-47a3-afc2-1db2e9c02e67 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.668108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003295-8629-437c-8e1a-51505ecc6864 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.673310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.674180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.674479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.674825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] PATH_INFO: `/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2/users/8fa3cc03ae39441caa65770b0c9bd3a1/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.677886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae16d3ac0c33436d93d1d32c4f691ac2, user_id=8fa3cc03ae39441caa65770b0c9bd3a1, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.685320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.698491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1042/2090] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:35 2022] PUT /identity/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2/users/8fa3cc03ae39441caa65770b0c9bd3a1/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:35.699326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.714167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.714830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.715145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.715421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.768709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:35.779454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26bc39-ea93-4208-a7aa-345cc07ab8d3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.784903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.786453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a9d9-6475-4256-b41a-4c36373d9d03 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.790530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.791698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.791698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.791898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.792418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.794449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.796113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1049/2091] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:35.797474 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.797774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.798066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.824727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1043/2092] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:35.826702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.838732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06b2a-35f9-4733-9de7-bd2fa8b3d2be admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.843496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] MFA Rules not processed for user `8fa3cc03ae39441caa65770b0c9bd3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:35.845587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.846340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.846560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.846767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.847120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.848753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.851437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.873141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1044/2093] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.874186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.894563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c280e-845c-4490-aa98-ca3715e7be2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.910479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.911292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.912125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.912625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.913574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.915940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.930860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1045/2094] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:35.935161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.946555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793eb66-8f8a-453d-b0ae-39d0caefb2dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.953547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1050/2095] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:35.954603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:35.955996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:35.961415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.962335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.962835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.963130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] PATH_INFO: `/v3/projects/343bf4b601ab47d0a76b5413a64f7e48/users/cc535000161d4ba0bca516d56407b3ec/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.980253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343bf4b601ab47d0a76b5413a64f7e48, user_id=cc535000161d4ba0bca516d56407b3ec, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.981616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e41e2-f098-4880-9f55-c32665508183 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:35.983168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:35.985690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:35.988178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:35.989157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:35.989465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:35.989770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] PATH_INFO: `/v3/users/8a754e9e71424c0aa514dd1d28b75fe7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:35.991237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a754e9e71424c0aa514dd1d28b75fe7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:35.994448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.013553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1046/2096] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:35 2022] PUT /identity/v3/projects/343bf4b601ab47d0a76b5413a64f7e48/users/cc535000161d4ba0bca516d56407b3ec/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:36.013553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:36.013553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.083698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a754e9e71424c0aa514dd1d28b75fe7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.096266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a754e9e71424c0aa514dd1d28b75fe7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.098286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a754e9e71424c0aa514dd1d28b75fe7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.102745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789f388-87dc-4197-a0a7-8cf2cd7e6a25 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:36.108138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:36.109617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.110009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.110440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.110554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a754e9e71424c0aa514dd1d28b75fe7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.127472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77874ea3-5ebf-4aff-acf7-346bc077f070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a754e9e71424c0aa514dd1d28b75fe7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.151772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1051/2097] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:35 2022] DELETE /identity/v3/users/8a754e9e71424c0aa514dd1d28b75fe7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:36.153382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.153678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.154053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.188294 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] MFA Rules not processed for user `cc535000161d4ba0bca516d56407b3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:36.190270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.203008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:36.204766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b6428-2243-4368-84ac-503e64e18c00 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.222801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1047/2098] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:36 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:36.225369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.225909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.226186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.249484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:36.252190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:36.253636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.296119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1052/2099] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:36.309941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:36.311618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.330462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1048/2100] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:36.333128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:36.338639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.383787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cc205-926a-4e58-bf9f-5a4eb5775223 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:36.387618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:36.388278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.388566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.388876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.390174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:36.399221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:36.400831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.402664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ae38f-d08a-44f8-ae91-d1fa981f9038 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:36.406959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:36.407611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.407900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.408209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.410081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.411169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1053/2101] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:36.412005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:36.413283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.475195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40230-6425-4bff-a3d3-a68371766635 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:36.479881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:36.480569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.480887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.481168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.481937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:36.483397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.496078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:36.497564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.499342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1049/2102] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:36.512801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1054/2103] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:36.835501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:36.837174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:36.882709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588f3-155e-4a97-84e4-a49f37a598cb service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:36.893692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:36.895640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:36.896121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:36.897079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] PATH_INFO: `/v3/projects/2950f7fdce854175b898675361e5c6ed` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:36.899215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2950f7fdce854175b898675361e5c6ed)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:36.902586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:36.970875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2950f7fdce854175b898675361e5c6ed', 'request_id': 'req-d791f7e0-d474-4011-bf96-0377c3da1c5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.979968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2950f7fdce854175b898675361e5c6ed', 'request_id': 'req-d791f7e0-d474-4011-bf96-0377c3da1c5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:36.988552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2950f7fdce854175b898675361e5c6ed', 'request_id': 'req-d791f7e0-d474-4011-bf96-0377c3da1c5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.001478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] The token cache is being invalidate because project 2950f7fdce854175b898675361e5c6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:37.001767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.002417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.004097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1050/2104] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:36 2022] DELETE /identity/v3/projects/2950f7fdce854175b898675361e5c6ed => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:37.008070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:37.025785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0895c-c93b-4999-891d-35028c650b90 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:37.089886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:37.090756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:37.091043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:37.091443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] PATH_INFO: `/v3/users/6e6f47a7a7c54d729fa12f846445af53` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:37.092682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6f47a7a7c54d729fa12f846445af53)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:37.094620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:37.126242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:37.153300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6f47a7a7c54d729fa12f846445af53'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.161867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6f47a7a7c54d729fa12f846445af53'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.162886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6f47a7a7c54d729fa12f846445af53'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.172425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6f47a7a7c54d729fa12f846445af53'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.187801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6f47a7a7c54d729fa12f846445af53'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.194982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1055/2105] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:37 2022] DELETE /identity/v3/users/6e6f47a7a7c54d729fa12f846445af53 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:37.604523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:37.606552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:37.621589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791f7e0-d474-4011-bf96-0377c3da1c5f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:37.698305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:37.699180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:37.699481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:37.699769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] PATH_INFO: `/v3/projects/57cb9c96406c4c1ebb9ed9073455daf5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:37.701003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cb9c96406c4c1ebb9ed9073455daf5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:37.702944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:37.731144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:37.755649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cb9c96406c4c1ebb9ed9073455daf5', 'request_id': 'req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.762645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cb9c96406c4c1ebb9ed9073455daf5', 'request_id': 'req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.770368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cb9c96406c4c1ebb9ed9073455daf5', 'request_id': 'req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.781163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] The token cache is being invalidate because project 57cb9c96406c4c1ebb9ed9073455daf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:37.781455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.782105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.783836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1051/2106] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:37 2022] DELETE /identity/v3/projects/57cb9c96406c4c1ebb9ed9073455daf5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:37.787865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:37.789348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:37.803034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e46cf3-9920-4ef0-be7e-bd8144b98332 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:37.867360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:37.868269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:37.868717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:37.869589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] PATH_INFO: `/v3/users/e9de8ba6348342bbbff47c6c8a66bb38` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:37.870862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9de8ba6348342bbbff47c6c8a66bb38)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:37.872879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:37.927649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9de8ba6348342bbbff47c6c8a66bb38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.937395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9de8ba6348342bbbff47c6c8a66bb38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.938209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9de8ba6348342bbbff47c6c8a66bb38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.946528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9de8ba6348342bbbff47c6c8a66bb38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.959079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-76e9a8f1-dc53-4b81-8649-5a43c43aaecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9de8ba6348342bbbff47c6c8a66bb38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:37.966878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1056/2107] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:37 2022] DELETE /identity/v3/users/e9de8ba6348342bbbff47c6c8a66bb38 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:38.303357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.304923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:38.321206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fb36-fcbf-4a44-8346-4fcf50e92e2c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.392034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.392934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.393219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.393497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] PATH_INFO: `/v3/projects/88e513aaca0846f3be39e7472670938f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.394691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e513aaca0846f3be39e7472670938f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.396640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.447380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e513aaca0846f3be39e7472670938f', 'request_id': 'req-74479b6d-0d0a-4a3b-8973-4817506788e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:38.453942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e513aaca0846f3be39e7472670938f', 'request_id': 'req-74479b6d-0d0a-4a3b-8973-4817506788e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:38.463522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e513aaca0846f3be39e7472670938f', 'request_id': 'req-74479b6d-0d0a-4a3b-8973-4817506788e1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:38.477687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] The token cache is being invalidate because project 88e513aaca0846f3be39e7472670938f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:38.478008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:38.479252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-74479b6d-0d0a-4a3b-8973-4817506788e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:38.481859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1052/2108] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:38 2022] DELETE /identity/v3/projects/88e513aaca0846f3be39e7472670938f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:38.486935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.487619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.487763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.507222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:38.508808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:38.522836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:38.576121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1057/2109] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:38.583846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.584133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.584409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.605026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:38.606168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:38.614395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1053/2110] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:38.618723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.629159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d41bf-34d6-480f-9905-e2d5a1d4a600 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.633620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.634305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.634805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.635081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.635818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:38.636138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.637758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.649493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1058/2111] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:38.654014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.663687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fa26f-9ddc-4d6d-91eb-2ab0b3f9f85c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.669122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.669762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.670045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.670317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.671011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.672982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.704772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1054/2112] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:38 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:38.708720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.719139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db183d-edce-4017-8a08-34439cd72b3b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.782709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.783413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.783701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.784053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.784851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.786762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.811937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1059/2113] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:38 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:38.813604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.828011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc0743-7737-4e5a-abfb-29f4a96d3eef admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.833080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.833750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.834065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.834496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.835277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.836929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.848565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1055/2114] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:38.853752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.865042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfe87-8a9a-4432-bcb3-3d24ffef24ea admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.870166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.870836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.871128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.871444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.872179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.873803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c0d4-064a-438d-a25b-2a9c0341c286 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.885897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1060/2115] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:38.889758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:38.894977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:38.901740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab015d-b56b-4e2d-9072-07d903878e85 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:38.909120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:38.910118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.910405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.910685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] PATH_INFO: `/v3/projects/ee3092be876549fa88015df159a61a42/users/27d4a7f7f0bf4a2fade9d5fd22c0f852/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:38.931027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3092be876549fa88015df159a61a42, user_id=27d4a7f7f0bf4a2fade9d5fd22c0f852, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:38.934419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:38.951017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1056/2116] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:38 2022] PUT /identity/v3/projects/ee3092be876549fa88015df159a61a42/users/27d4a7f7f0bf4a2fade9d5fd22c0f852/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:38.961724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:38.962022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:38.962834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:39.003239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] MFA Rules not processed for user `27d4a7f7f0bf4a2fade9d5fd22c0f852`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:39.008779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:39.114899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1061/2117] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:39.130142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:39.136759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:39.198850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b7644-53bc-46c7-a323-f1b3f049d3b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:39.202574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:39.203233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:39.203501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:39.203778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:39.213124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:39.214837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57619b64-8ca3-48c3-97ae-8bcc3e0863ab service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:39.226996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1057/2118] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:39.262170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:39.277706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:39.334320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4df32-a82c-45c5-8b40-9f890471fd2c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:39.339829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:39.340760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:39.341047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:39.341327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] PATH_INFO: `/v3/users/27d4a7f7f0bf4a2fade9d5fd22c0f852` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:39.342604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d4a7f7f0bf4a2fade9d5fd22c0f852)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:39.344703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:39.416762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d4a7f7f0bf4a2fade9d5fd22c0f852'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:39.426144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d4a7f7f0bf4a2fade9d5fd22c0f852'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:39.427005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d4a7f7f0bf4a2fade9d5fd22c0f852'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:39.435984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d4a7f7f0bf4a2fade9d5fd22c0f852'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:39.458884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d4a7f7f0bf4a2fade9d5fd22c0f852'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:39.467954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1062/2119] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:39 2022] DELETE /identity/v3/users/27d4a7f7f0bf4a2fade9d5fd22c0f852 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:39.472208 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:39.472721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:39.472827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:39.493268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:39.496841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:39.580702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1058/2120] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:39.595834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:39.598500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:39.722996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd297a7-b07d-400d-8c2c-2024773e8b59 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:39.727669 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:39.728652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:39.728960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:39.729525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:39.741094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:39.742375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:39.753881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1063/2121] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:40.111047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.113363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.132915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2c57-490e-48f0-97e5-459849df4b7f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.154118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.155226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.155551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.156064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] PATH_INFO: `/v3/users/980d1f84ba3549e3b79ddbff8e090538` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.157621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=980d1f84ba3549e3b79ddbff8e090538)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.160528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.233901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980d1f84ba3549e3b79ddbff8e090538'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.243517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980d1f84ba3549e3b79ddbff8e090538'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.245062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980d1f84ba3549e3b79ddbff8e090538'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.245836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980d1f84ba3549e3b79ddbff8e090538'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.259253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980d1f84ba3549e3b79ddbff8e090538'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.262014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:40.277763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1059/2122] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:40 2022] DELETE /identity/v3/users/980d1f84ba3549e3b79ddbff8e090538 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:40.361471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.363602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.385786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617c8c-66f5-496d-ad37-1b5b263709ed service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.442932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:40.506927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.508587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.508913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.509230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] PATH_INFO: `/v3/projects/ee3092be876549fa88015df159a61a42` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.510607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3092be876549fa88015df159a61a42)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.513068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.526097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.528670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.557946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bed19-de18-4e3f-89bd-5a6ccd1a13ad admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.565048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.565920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.566371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.566659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] PATH_INFO: `/v3/projects/2add5497070940cd8e911a8e6274c73c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.568831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2add5497070940cd8e911a8e6274c73c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.577112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.604224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3092be876549fa88015df159a61a42', 'request_id': 'req-85e72e77-3fda-4800-8d2b-2b006c8286b9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.610943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3092be876549fa88015df159a61a42', 'request_id': 'req-85e72e77-3fda-4800-8d2b-2b006c8286b9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.619037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3092be876549fa88015df159a61a42', 'request_id': 'req-85e72e77-3fda-4800-8d2b-2b006c8286b9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.631903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] The token cache is being invalidate because project ee3092be876549fa88015df159a61a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:40.632244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.633846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-85e72e77-3fda-4800-8d2b-2b006c8286b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.635893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1064/2123] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:40 2022] DELETE /identity/v3/projects/ee3092be876549fa88015df159a61a42 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:40.639200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2add5497070940cd8e911a8e6274c73c', 'request_id': 'req-00b10f38-3b55-4dee-84b6-ab38244e816c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.641041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f3588-2091-4b23-bda9-573e1f4a5d7f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.641449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f3588-2091-4b23-bda9-573e1f4a5d7f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.641818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f3588-2091-4b23-bda9-573e1f4a5d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.644425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2add5497070940cd8e911a8e6274c73c', 'request_id': 'req-00b10f38-3b55-4dee-84b6-ab38244e816c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.654889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2add5497070940cd8e911a8e6274c73c', 'request_id': 'req-00b10f38-3b55-4dee-84b6-ab38244e816c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.663559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-732f3588-2091-4b23-bda9-573e1f4a5d7f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:40.664826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-732f3588-2091-4b23-bda9-573e1f4a5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.672056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] The token cache is being invalidate because project 2add5497070940cd8e911a8e6274c73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:40.672485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.673669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00b10f38-3b55-4dee-84b6-ab38244e816c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:40.675707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1060/2124] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:40 2022] DELETE /identity/v3/projects/2add5497070940cd8e911a8e6274c73c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:40.680852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5999d-47f7-4a1d-8a6b-707196988eda None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.681030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5999d-47f7-4a1d-8a6b-707196988eda None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.681978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5999d-47f7-4a1d-8a6b-707196988eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.704132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5ae5999d-47f7-4a1d-8a6b-707196988eda None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:40.706038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ae5999d-47f7-4a1d-8a6b-707196988eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.744659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1065/2125] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:40.752486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.752825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.753216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.776297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:40.777612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.780438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1061/2126] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:40.785884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1066/2127] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:40.788007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.788368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.788817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.789792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.799271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a790f-43b6-4377-a474-a739ef2b9bf6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.804217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.804955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.805435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.805552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.806860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:40.806860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.808536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.808670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:40.809780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:40.818209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1062/2128] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:40.823108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.824936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1067/2129] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:40.828501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.832433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b97f39-da30-4149-876f-c9264264b58d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.837305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.838603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.838824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.839104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.839421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d0d2-d605-445e-aa08-9a0e75853255 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.839834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:40.840140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.841776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.844203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.844883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.845172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.845480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.846177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.848059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.853804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1063/2130] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:40.858069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.868163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f0ac-8ed9-458e-b0fd-c058acd51528 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.873593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.874258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.874544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.874824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.874948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1068/2131] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:40.875514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.877458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:40.878963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.910302 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:40.916157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1064/2132] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:40.917123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:40.949936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dae45f-8081-48e2-a4a2-ba57d3ade162 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.956990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:40.984138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdb6ac-623a-435b-bf02-a958a4dff648 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:40.988955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:40.989733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:40.990104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:40.990839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:40.991620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:40.994557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.009757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.010481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.010786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.011150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.011814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.013853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.027385 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1065/2133] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:41.032270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.043186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1069/2134] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:40 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:41.043609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8b33-523e-4eef-8ba4-a2ea40a434cc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.047322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.048930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.049592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.049938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.050222 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.050952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.052556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.058717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f53093-053d-44df-bb03-336c9179e5f8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.063753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.064350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1066/2135] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:41.064876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.065174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.065456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.066193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.067885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.068058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.078281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67169a81-fef2-4a80-92e6-848a04b2bb20 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.083055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1070/2136] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:41.084412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.086845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.088839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.089188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.089465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.090201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.091866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.095429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142aa0f5-c699-4815-9a7b-08f873cfe3a2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.100199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.101139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.101428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.101708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.104674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.104674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.105046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1067/2137] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:41.110759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.120255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1071/2138] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:41.123098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e769d1-2f64-4359-90f7-f33acd3e236c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.123755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.128261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.129281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.129626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.130143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] PATH_INFO: `/v3/projects/028915500ec242cab30066e4717f02ef/users/314a24145bc448e0b23dfd88968b46bf/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.136968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635948e7-be24-426a-a8a0-2f673a01556d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.143539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.143974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.144258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.144867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] PATH_INFO: `/v3/projects/4b09c15f97154b0b9cd5a9e80e9e54a6/users/20c7cf1573374f36ad2d09b78ab968e8/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.147733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=028915500ec242cab30066e4717f02ef, user_id=314a24145bc448e0b23dfd88968b46bf, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.150930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b0cb1-75ea-4f1c-8c11-db81671916b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.164566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b09c15f97154b0b9cd5a9e80e9e54a6, user_id=20c7cf1573374f36ad2d09b78ab968e8, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.167685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c12471-367e-46db-8dd3-80196050eff6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.172729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1068/2139] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:41 2022] PUT /identity/v3/projects/028915500ec242cab30066e4717f02ef/users/314a24145bc448e0b23dfd88968b46bf/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:41.178890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.179589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.179876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.184498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1072/2140] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:41 2022] PUT /identity/v3/projects/4b09c15f97154b0b9cd5a9e80e9e54a6/users/20c7cf1573374f36ad2d09b78ab968e8/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:41.194559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.194932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.195248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.210211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] MFA Rules not processed for user `314a24145bc448e0b23dfd88968b46bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:41.211427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:41.224748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] MFA Rules not processed for user `20c7cf1573374f36ad2d09b78ab968e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:41.226440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:41.299334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1069/2141] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:41.303869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.312187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1073/2142] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:41.323349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.324741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:41.331318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:41.367519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368e407-f9ac-436c-8daf-1b8415c4d973 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.373328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.373994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.374278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.374553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.375362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.377335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.382857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e42455-6089-4af8-a8a8-59695605e9e0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.387544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.388225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.388510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.388817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.395213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:41.398278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.400295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.409999 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1074/2143] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:41.413488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1070/2144] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:41 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:41.414436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.425515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ba69-9e74-4d82-a612-9a289799aded service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.489891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.490555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.490916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.491246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.492096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.494839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.516409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1075/2145] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:41 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:41.520434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.531437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c8d9-71f7-4229-89cb-07d840062aeb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.537307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.537986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.538348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.538655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.539433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.544345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.556747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1071/2146] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:41.560838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.573282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7b961-a849-4b1d-9ba3-00196be1f071 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.579350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.580378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.580770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.581122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] PATH_INFO: `/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.598860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8394df60b8f45e687cdb60e9eea5b1f, user_id=9f95946a3eec4c4ea0e9b3a5f7bd6fc6, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.607388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.626248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1076/2147] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:41 2022] PUT /identity/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:41.630470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.648025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aa116-b3f9-4d2f-b902-83ce680dce64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.743494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.752888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.752888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.752888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.752888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.752888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4a87b-fbf6-41de-920a-ed52d8649800 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.766731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1072/2148] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:41.770901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.787698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7bd6-85a0-43d2-b8c3-b08f794f46eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:41.797190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:41.798449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.798936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.799704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] PATH_INFO: `/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.802665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8394df60b8f45e687cdb60e9eea5b1f, user_id=9f95946a3eec4c4ea0e9b3a5f7bd6fc6, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:41.808005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:41.825093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1077/2149] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:41 2022] PUT /identity/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:41.837122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:41.838314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:41.838789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:41.875942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] MFA Rules not processed for user `9f95946a3eec4c4ea0e9b3a5f7bd6fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:41.877186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:41.944147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1073/2150] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:41.955659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:41.957465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:42.019110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a744120-ea09-4590-829f-32943d94f6fa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:42.022761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:42.023419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:42.023705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:42.023985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:42.034457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:42.035903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:42.047205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1078/2151] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:42.119754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:42.129818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1800e4-c7b6-4238-af76-ddf60b533b4b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:42.133397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:42.134033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:42.134965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:42.135244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:42.191851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:42.193350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea56ec-b468-491b-9b38-3a34e1a304dc service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:42.202872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1074/2152] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:42 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:44.212992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:44.212992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.218395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:44.295259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5588fd-aca2-4e39-8285-f3058d2be743 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:44.299832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:44.300729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.301014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.301285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] PATH_INFO: `/v3/users/60756d1f09274d1cb6d395802c44895a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.302852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60756d1f09274d1cb6d395802c44895a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:44.304809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:44.359091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60756d1f09274d1cb6d395802c44895a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.371775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60756d1f09274d1cb6d395802c44895a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.372423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60756d1f09274d1cb6d395802c44895a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.380866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60756d1f09274d1cb6d395802c44895a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.393671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60756d1f09274d1cb6d395802c44895a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.400312 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1079/2153] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:44 2022] DELETE /identity/v3/users/60756d1f09274d1cb6d395802c44895a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:44.405236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.405721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.405891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.411769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:44.429515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:44.431923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.441817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:44.443273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.517057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1075/2154] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:44.522374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da8687-470b-4f1a-adf7-cf459a2585b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:44.528094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:44.528236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:44.529080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.529368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.529502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.529649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] PATH_INFO: `/v3/users/78295aa21c2c4517aff1bbbb6f931f4e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.530870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78295aa21c2c4517aff1bbbb6f931f4e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:44.532831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:44.556287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:44.584127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78295aa21c2c4517aff1bbbb6f931f4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.592857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78295aa21c2c4517aff1bbbb6f931f4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.593035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e084d-b86d-48b6-a12a-c7b92f47c661 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:44.593607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78295aa21c2c4517aff1bbbb6f931f4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.596680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:44.597336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.597631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.597912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.601136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78295aa21c2c4517aff1bbbb6f931f4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.607671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:44.609323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e6be-70b5-41fb-bfe2-11ead727384f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:44.613363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78295aa21c2c4517aff1bbbb6f931f4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:44.619342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1076/2155] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:44.621452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1080/2156] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:44 2022] DELETE /identity/v3/users/78295aa21c2c4517aff1bbbb6f931f4e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:44.625664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.625967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.626251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.648921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:44.650815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.727888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1077/2157] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:44.736761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:44.738214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:44.812527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917c7c5-d8b4-46de-a84c-f81d88939393 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:44.820425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:44.821469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:44.821941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:44.822431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:44.836019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:44.837306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:44.849535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1081/2158] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:45.051415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:45.054912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:45.084921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80929b-66dd-49f8-80d2-3f7ba9b555c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:45.094063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:45.095218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:45.095478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:45.095880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] PATH_INFO: `/v3/projects/f1290ce33e624a0e9f3570a7769f745f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:45.098106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1290ce33e624a0e9f3570a7769f745f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:45.100804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:45.164774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:45.173758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1290ce33e624a0e9f3570a7769f745f', 'request_id': 'req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.183457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1290ce33e624a0e9f3570a7769f745f', 'request_id': 'req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.193969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1290ce33e624a0e9f3570a7769f745f', 'request_id': 'req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.208654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] The token cache is being invalidate because project f1290ce33e624a0e9f3570a7769f745f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:45.208654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.208654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.211213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:45.212713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1078/2159] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:45 2022] DELETE /identity/v3/projects/f1290ce33e624a0e9f3570a7769f745f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:45.212946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:45.214757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:45.216221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:45.252771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71717d-feae-4436-ba76-d1127a03ba17 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:45.297967 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef2cf-7f18-45ae-a6a5-d3d15a2bf578 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:45.302820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:45.303705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:45.303992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:45.304273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] PATH_INFO: `/v3/users/8fa3cc03ae39441caa65770b0c9bd3a1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:45.305677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa3cc03ae39441caa65770b0c9bd3a1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:45.308499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:45.324724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:45.325619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:45.325916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:45.326198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] PATH_INFO: `/v3/projects/96a2d3d39f2a4920931044f8ee23d87b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:45.327609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a2d3d39f2a4920931044f8ee23d87b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:45.330291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:45.374914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa3cc03ae39441caa65770b0c9bd3a1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.384743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa3cc03ae39441caa65770b0c9bd3a1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.386340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa3cc03ae39441caa65770b0c9bd3a1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.387054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa3cc03ae39441caa65770b0c9bd3a1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.394165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:45.397350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a2d3d39f2a4920931044f8ee23d87b', 'request_id': 'req-84d20949-05e1-4bd4-9a54-fb1695e83bfd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.402157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa3cc03ae39441caa65770b0c9bd3a1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.410984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a2d3d39f2a4920931044f8ee23d87b', 'request_id': 'req-84d20949-05e1-4bd4-9a54-fb1695e83bfd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.420392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1079/2160] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:45 2022] DELETE /identity/v3/users/8fa3cc03ae39441caa65770b0c9bd3a1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:45.424744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a2d3d39f2a4920931044f8ee23d87b', 'request_id': 'req-84d20949-05e1-4bd4-9a54-fb1695e83bfd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.439187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] The token cache is being invalidate because project 96a2d3d39f2a4920931044f8ee23d87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:45.439490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.440175 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.441986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1082/2161] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:45 2022] DELETE /identity/v3/projects/96a2d3d39f2a4920931044f8ee23d87b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:45.445690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:45.448239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:45.545338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:45.565689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de65b44-6c72-439d-847f-bf4e9fc70b51 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:45.574523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:45.575576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:45.575898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:45.576200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] PATH_INFO: `/v3/users/5aa26f9162c742aeb2be440c98bd177e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:45.577521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa26f9162c742aeb2be440c98bd177e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:45.579506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:45.635815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa26f9162c742aeb2be440c98bd177e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.643725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa26f9162c742aeb2be440c98bd177e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.644454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa26f9162c742aeb2be440c98bd177e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.645138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa26f9162c742aeb2be440c98bd177e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.657989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7fd20b6-8524-478f-80c8-8cc95ccc5191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa26f9162c742aeb2be440c98bd177e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.676894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1080/2162] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:45 2022] DELETE /identity/v3/users/5aa26f9162c742aeb2be440c98bd177e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:45.753448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:45.758555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:45.849457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20949-05e1-4bd4-9a54-fb1695e83bfd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:45.910794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:45.911700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:45.911993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:45.912268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] PATH_INFO: `/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:45.913508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae16d3ac0c33436d93d1d32c4f691ac2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:45.915444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:45.972788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae16d3ac0c33436d93d1d32c4f691ac2', 'request_id': 'req-bf8fd694-a845-48de-9778-68dbd1324bcd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.979806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae16d3ac0c33436d93d1d32c4f691ac2', 'request_id': 'req-bf8fd694-a845-48de-9778-68dbd1324bcd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.987738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae16d3ac0c33436d93d1d32c4f691ac2', 'request_id': 'req-bf8fd694-a845-48de-9778-68dbd1324bcd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:45.999747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] The token cache is being invalidate because project ae16d3ac0c33436d93d1d32c4f691ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:46.000010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.000674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.002222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1083/2163] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:45 2022] DELETE /identity/v3/projects/ae16d3ac0c33436d93d1d32c4f691ac2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:46.007945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92f005-fd8b-46c5-8666-baff786acdf1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.008239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92f005-fd8b-46c5-8666-baff786acdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.008534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92f005-fd8b-46c5-8666-baff786acdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.039335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.040729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dd92f005-fd8b-46c5-8666-baff786acdf1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.040921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.042707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dd92f005-fd8b-46c5-8666-baff786acdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.059338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:46.131064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1081/2164] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:46.139517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.139796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.140004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.140878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd694-a845-48de-9778-68dbd1324bcd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.145725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.146590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.146898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.147177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] PATH_INFO: `/v3/projects/107a05c87ead480e9e58687f96e410e1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.149056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=107a05c87ead480e9e58687f96e410e1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.151829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.158525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.159746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.172550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1082/2165] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:46.176411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.187455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837dc38-acb5-487f-b532-8af7813624fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.205778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107a05c87ead480e9e58687f96e410e1', 'request_id': 'req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.212129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107a05c87ead480e9e58687f96e410e1', 'request_id': 'req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.220570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107a05c87ead480e9e58687f96e410e1', 'request_id': 'req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.240595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] The token cache is being invalidate because project 107a05c87ead480e9e58687f96e410e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:46.240915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.241602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3648b3a1-84c6-4e64-85d3-a49b4ec582ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:46.243271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1084/2166] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:46 2022] DELETE /identity/v3/projects/107a05c87ead480e9e58687f96e410e1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:46.249266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0b2a-63e7-4429-befa-fd4db2353de8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.249564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0b2a-63e7-4429-befa-fd4db2353de8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.249851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0b2a-63e7-4429-befa-fd4db2353de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.254586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.255246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.255528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.255804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.256676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:46.256989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.258594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.261089 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:46.269179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70bc0b2a-63e7-4429-befa-fd4db2353de8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.270514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70bc0b2a-63e7-4429-befa-fd4db2353de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.272315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1083/2167] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:02:46.276563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.278229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.280266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1085/2168] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:46.287835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.288126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.288413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.290625 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1a2bf-5af3-4ee0-bd74-e8177affa80d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.295905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.297484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.297484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.297484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.298278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.300295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.312010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.313334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.321959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1086/2169] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:46.326594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.331092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1084/2170] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.334658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.335643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538288d2-031e-4ab9-970a-abe0e2a02c02 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.396858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.397443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.397574 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b124e2-0ea7-480e-b307-1498a4d9e9c8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.397725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.398000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.398733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:46.399040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.401225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.402990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.403740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.404031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.404313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.405053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.407106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.419689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1087/2171] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:02:46.421444 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.432298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1085/2172] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:46.433935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.434762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68413753-7044-45d5-8d68-154e948d9356 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.439383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.440055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.440345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.440639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.441399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.443369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.443645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c683a5-a56d-47a7-b1b7-54e37c5be1f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.448962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.449613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.449927 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.450210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.450966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.452623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.462923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1086/2173] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:46.465617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:46.468161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.473786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1088/2174] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.477342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.479923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026f14-18f5-4453-9892-48d8b5955b26 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.542906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.543576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.543864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.544137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.546407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.546540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.547393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da5d0-8b95-4362-8c30-83d01230061c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.558525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.558822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1087/2175] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:46.559480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.559836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.561609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.562937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.563050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.564941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.575843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713817a7-09ca-4e0a-8db5-9b7f3e0acecc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.581453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.582874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.583240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.583524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] PATH_INFO: `/v3/projects/64a7092266f54fc68a03430b81f45b12/users/c8b0b4f1021c4663a9ffd1621402dc32/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.599198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1089/2176] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.603461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.614945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a7092266f54fc68a03430b81f45b12, user_id=c8b0b4f1021c4663a9ffd1621402dc32, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.616319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e21c7-b96e-4953-8551-e70eed09a64a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.617681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379380d-e30e-472f-88d4-edf93caec389 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.629938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.630682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.630967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.631273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.632113 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.633998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.637107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1088/2177] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:46 2022] PUT /identity/v3/projects/64a7092266f54fc68a03430b81f45b12/users/c8b0b4f1021c4663a9ffd1621402dc32/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:46.645830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1090/2178] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:46.649291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.650047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.650346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.650686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.660169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993747-b264-4cd2-ab19-e19d8f58b848 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.677377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] MFA Rules not processed for user `c8b0b4f1021c4663a9ffd1621402dc32`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.678589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.693214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:46.723718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.724387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.724702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.724982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.725974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.727565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.741719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1091/2179] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:46.742911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.754158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9684-546a-4d04-a3a4-069da3a54705 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.758126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1089/2180] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:46.759441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.761059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.761356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.762016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] PATH_INFO: `/v3/projects/8cec85db15464c8bb1a7df47a97af66b/users/b7a7499120da44acb23ce8fdb136d215/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.762784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.773601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9558afb-7103-403d-adbd-d98cddc1edbd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.778180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cec85db15464c8bb1a7df47a97af66b, user_id=b7a7499120da44acb23ce8fdb136d215, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.778304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.778861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.779148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.779426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.780120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.780879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af686da-ace1-44bd-bac4-37896147a0ef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.782081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.795913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1092/2181] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:46 2022] PUT /identity/v3/projects/8cec85db15464c8bb1a7df47a97af66b/users/b7a7499120da44acb23ce8fdb136d215/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:46.806229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.806518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.806844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.821752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1090/2182] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.823060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.834748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] MFA Rules not processed for user `b7a7499120da44acb23ce8fdb136d215`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:46.835962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:46.890117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:46.892159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907b658-3649-41ca-a93b-d77623534fba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.897471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.898173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.898465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.898794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.899611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.901697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.916809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1093/2183] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:46.921140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.925855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1091/2184] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.926935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.933356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5723b78-3541-43a6-9552-dcf664c02c32 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.937587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef3ff1-e62a-44fb-80e0-2f3a4517ee0b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.937968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.938639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.938909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.939187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.939905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.941824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.942283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:46.942928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:46.943215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:46.943496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:46.944224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:46.945898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:46.956657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1092/2185] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:46.961669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:46.968575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1094/2186] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:46.971426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f775223-c634-4401-8534-69a0c516ff9e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:46.972458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.041084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.042068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118a15-e48a-45d4-9743-e1f520b82327 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.042805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.043091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.043371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.046982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.047303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.047995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.048283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.048558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.049300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.049408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.051220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.061173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1093/2187] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:47.065736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.071835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1095/2188] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:46 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:47.075758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.078878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bdf14-49bf-4d32-b08b-7576c27ce8d0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.083634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.084496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.084807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.085249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] PATH_INFO: `/v3/projects/aba64a0528904fcc80227eae89cd2600/users/7fd0942062b84621b4f6eb66992f3229/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.085359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0566c9-9e40-4acb-a31f-51b881810b6d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.087466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:47.091177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.091788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.092099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.092376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.093128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.094725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.103484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba64a0528904fcc80227eae89cd2600, user_id=7fd0942062b84621b4f6eb66992f3229, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.109504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1096/2189] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:47.109640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35478dd-a81f-4003-a569-b9488b8116f2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.110744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.123754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b2581-2a2d-486b-a4f9-aeadad5c3947 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.130503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1094/2190] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:47 2022] PUT /identity/v3/projects/aba64a0528904fcc80227eae89cd2600/users/7fd0942062b84621b4f6eb66992f3229/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:47.142498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.142795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.143087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.180601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] MFA Rules not processed for user `7fd0942062b84621b4f6eb66992f3229`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:47.183942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:47.211401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.212297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.212587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.212917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] PATH_INFO: `/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7/users/86b7963c3b974b609aa3a8c4af2be983/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.229647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8737ff600a49f8b8875ccd3048c9d7, user_id=86b7963c3b974b609aa3a8c4af2be983, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.232343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.246659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1097/2191] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:47 2022] PUT /identity/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7/users/86b7963c3b974b609aa3a8c4af2be983/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:47.250573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.267785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1095/2192] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:47.271938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.275264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:47.283798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a20668-e3d9-4ff5-b455-b44a29c71aa9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.325411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a01fb7-211d-4a52-b2bd-cfd0e1650acf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.330174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.330839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.331131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.331410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.332147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.333783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.343797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1098/2193] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:47.348451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.358913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7a1e6-8b20-40b8-9d2f-817339a51a50 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.359569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.360230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.360511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.360814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.361504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.363407 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.363811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.364706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.364995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.365277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] PATH_INFO: `/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7/users/86b7963c3b974b609aa3a8c4af2be983/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.367230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8737ff600a49f8b8875ccd3048c9d7, user_id=86b7963c3b974b609aa3a8c4af2be983, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.370490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04128c-e1b5-45d2-9304-5f9b85d66c57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.392717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1099/2194] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:47 2022] PUT /identity/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7/users/86b7963c3b974b609aa3a8c4af2be983/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:47.394862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.395278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.395637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.396313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1096/2195] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:47 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:47.400299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.425369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] MFA Rules not processed for user `86b7963c3b974b609aa3a8c4af2be983`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:47.426593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:47.460993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:47.465762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7dda5-56b7-4e8e-b273-cac529fe7757 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.471787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.472463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.472780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.473202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.473905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.476175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.501987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1097/2196] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:47 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:47.503498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.508212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1100/2197] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:47.514272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdf1db-9aa6-4782-9cab-e64cdc624e5c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.515743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.517157 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:47.520046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.521644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.521836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.522308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.523621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.526806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.538182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1098/2198] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:47.543240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.552940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d5585-3fe1-4e3c-bc8b-131b8673d897 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.558219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.559075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.559354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.559629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] PATH_INFO: `/v3/projects/624c74557d2848d68847e844b67c3a9a/users/54b404d7459e4ddfb897ce4faea57656/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.575148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=624c74557d2848d68847e844b67c3a9a, user_id=54b404d7459e4ddfb897ce4faea57656, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.578326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5a704-97a7-4a74-9cea-8f73f685ea0d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.578488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.581665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.582379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.582599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.582900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.593289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.593455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1099/2199] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:47 2022] PUT /identity/v3/projects/624c74557d2848d68847e844b67c3a9a/users/54b404d7459e4ddfb897ce4faea57656/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:47.594907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.597366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.607196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1101/2200] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:47.664720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.674603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7eba-c238-4d4e-aa42-2ceb55c06317 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.712711 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911fa8-e17d-4be6-849f-eadba6f9e37e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.714225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:47.720442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.721255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.721613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.721956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.722740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.724489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.736338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1100/2201] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:47.740325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.740456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.741196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.741546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.741887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.751113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d7c8-ef9c-4ac0-b31f-950b81bcd900 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.756044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.756928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.757212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.757485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] PATH_INFO: `/v3/projects/624c74557d2848d68847e844b67c3a9a/users/54b404d7459e4ddfb897ce4faea57656/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.759450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624c74557d2848d68847e844b67c3a9a, user_id=54b404d7459e4ddfb897ce4faea57656, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.762430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904398b0-7032-43ef-bffc-cb5136d25451 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.780088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1101/2202] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:47 2022] PUT /identity/v3/projects/624c74557d2848d68847e844b67c3a9a/users/54b404d7459e4ddfb897ce4faea57656/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:02:47.790088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.790431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.790658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.803535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.805341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.816044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1102/2203] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:02:47.819554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] MFA Rules not processed for user `54b404d7459e4ddfb897ce4faea57656`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:47.820791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:47.896007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1102/2204] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:47.908179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:47.909695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:47.921713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ccd7d-32cd-4a81-a82a-6204d425c619 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:47.975978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:47.976722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:47.977072 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:47.977415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:47.987752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:47.989222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:47.999413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1103/2205] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:48.050124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:48.061398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c004c-be86-4643-8ac5-89b451ca2740 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:48.065126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:48.065859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:48.066209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:48.066558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:48.085359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:48.143689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:48.145257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fd65d-1b88-4246-ac1b-2787fe9fe794 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:48.164686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1103/2206] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:51.221739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:51.285945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b674b-8b68-45f4-a84d-533f7cfff9d6 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:51.291995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:51.293005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:51.293290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:51.293567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] PATH_INFO: `/v3/users/b7a7499120da44acb23ce8fdb136d215` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:51.294783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a7499120da44acb23ce8fdb136d215)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:51.296745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:51.335027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:51.349915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a7499120da44acb23ce8fdb136d215'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.358164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a7499120da44acb23ce8fdb136d215'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.358845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a7499120da44acb23ce8fdb136d215'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.367188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a7499120da44acb23ce8fdb136d215'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.380739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a7499120da44acb23ce8fdb136d215'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.387418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1104/2207] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:51 2022] DELETE /identity/v3/users/b7a7499120da44acb23ce8fdb136d215 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:51.391742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:51.392249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:51.392430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:51.410723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:51.412107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:51.481978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1104/2208] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:51.491888 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:51.493463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:51.574171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b7c4-b246-40fe-a139-8bbb1bb9ee69 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:51.578056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:51.578705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:51.578992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:51.579267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:51.589678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:51.591104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:51.601130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1105/2209] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:51.880972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:51.882396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:51.900532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ef7c7-d551-46b2-b886-3659459099ac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:51.906153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:51.907021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:51.907309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:51.907583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] PATH_INFO: `/v3/projects/8cec85db15464c8bb1a7df47a97af66b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:51.908813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cec85db15464c8bb1a7df47a97af66b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:51.910741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:51.962527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cec85db15464c8bb1a7df47a97af66b', 'request_id': 'req-5979095d-dfad-4950-adda-956a4fcfa888'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.969690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cec85db15464c8bb1a7df47a97af66b', 'request_id': 'req-5979095d-dfad-4950-adda-956a4fcfa888'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.981998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cec85db15464c8bb1a7df47a97af66b', 'request_id': 'req-5979095d-dfad-4950-adda-956a4fcfa888'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.996608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] The token cache is being invalidate because project 8cec85db15464c8bb1a7df47a97af66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:51.997225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:51.998108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.000020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1105/2210] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:51 2022] DELETE /identity/v3/projects/8cec85db15464c8bb1a7df47a97af66b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:52.004212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.020834 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdffe-91c1-4e6d-b9eb-000208e71231 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.111375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.112249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.112537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.112850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] PATH_INFO: `/v3/users/86b7963c3b974b609aa3a8c4af2be983` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.114047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b7963c3b974b609aa3a8c4af2be983)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:52.115985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:52.170757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b7963c3b974b609aa3a8c4af2be983'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.174263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:52.178119 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b7963c3b974b609aa3a8c4af2be983'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.179185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b7963c3b974b609aa3a8c4af2be983'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.187893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b7963c3b974b609aa3a8c4af2be983'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.204777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db7b6af2-4098-4c43-9a67-cd86026fd1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b7963c3b974b609aa3a8c4af2be983'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.210734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1106/2211] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:52 2022] DELETE /identity/v3/users/86b7963c3b974b609aa3a8c4af2be983 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:52.482651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.484186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:52.498235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979095d-dfad-4950-adda-956a4fcfa888 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.547056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:52.566473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.567337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.567619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.567892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] PATH_INFO: `/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.569091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8737ff600a49f8b8875ccd3048c9d7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:52.571004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:52.628443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8737ff600a49f8b8875ccd3048c9d7', 'request_id': 'req-36673189-97c9-4dd7-926e-8eec606dea65'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.634564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8737ff600a49f8b8875ccd3048c9d7', 'request_id': 'req-36673189-97c9-4dd7-926e-8eec606dea65'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.642127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8737ff600a49f8b8875ccd3048c9d7', 'request_id': 'req-36673189-97c9-4dd7-926e-8eec606dea65'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.653254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] The token cache is being invalidate because project 9f8737ff600a49f8b8875ccd3048c9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:52.653564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.654179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-36673189-97c9-4dd7-926e-8eec606dea65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:52.655697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1106/2212] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:52 2022] DELETE /identity/v3/projects/9f8737ff600a49f8b8875ccd3048c9d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:52.660501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.661012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.661109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.681544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:52.682747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:52.749700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1107/2213] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:52.757555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.757842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.758117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.777818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:52.779015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:52.786695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1107/2214] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:52.790943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.802032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d139-8161-4dbe-b96d-86a69e9ae9d4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.808574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.809409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.809704 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.810121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.810851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:52.811190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:52.812873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:52.825847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1108/2215] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:02:52.830350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.840221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9fef6-a4d8-4580-bde2-c0fccdfcf5bd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.844844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.845486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.845766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.846038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.846721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:52.848642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:52.873381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1108/2216] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:52 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:52.877503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.889438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd9359-f183-45d4-8139-b0f997ac75cb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.949804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.950612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.950973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.951324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.952204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:52.954167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:52.975698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1109/2217] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:52 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:52.980402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:52.990676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc9dc-cd86-4c2f-9f77-526bc8ec6540 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:52.996738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:52.997452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:52.997738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:52.998015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:52.998747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.000368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.011281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1109/2218] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:53.015427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.026842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76027b-0ebb-47e0-b78e-fa1dae6cb3d5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.031948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.032735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.033446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.033584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.034746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.037481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b65cf5-3b9d-4315-adec-0fd619fc787a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.048995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1110/2219] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:53.053300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.063763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df065244-2c10-4960-98d4-48c3e14a4082 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.068265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.069159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.069444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.069722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] PATH_INFO: `/v3/projects/f5f6ff20978445c28a479e8a08eda113/users/9798e7e922b347e58fc118eaa788154b/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.084570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f6ff20978445c28a479e8a08eda113, user_id=9798e7e922b347e58fc118eaa788154b, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.087302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.103122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1110/2220] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:53 2022] PUT /identity/v3/projects/f5f6ff20978445c28a479e8a08eda113/users/9798e7e922b347e58fc118eaa788154b/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:53.115140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.115497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.116036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.147288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] MFA Rules not processed for user `9798e7e922b347e58fc118eaa788154b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:53.148532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:53.178530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:53.241055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1111/2221] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:53.246438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.312873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ddba-228b-4224-8ee0-f44ce0b0cbd2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.317800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.318469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.318755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.319034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.319746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.321751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.335748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:53.360541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1111/2222] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:53 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:53.361705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.373716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f29916-74b7-4a39-8bb2-62601f13a8bd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.435845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.436531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.436846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.437125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.437995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.439979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.462144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1112/2223] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:53 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:53.466797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.477440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7c9eb-19e6-4a73-9384-993a2e7270bb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.482086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.482741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.483274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.483274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.484010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.485664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.496200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1112/2224] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:53.500149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.510645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ac7a2-3e2b-4906-a73b-bd3b5ca9a6cf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.515222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.516087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.516372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.516687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] PATH_INFO: `/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b/users/9f82cc9175b94fa68ebfcc6a0fabf5a3/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.531783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b8df2a2edd4fd9847caf73ce44461b, user_id=9f82cc9175b94fa68ebfcc6a0fabf5a3, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.534447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.549645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1113/2225] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:53 2022] PUT /identity/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b/users/9f82cc9175b94fa68ebfcc6a0fabf5a3/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:53.553509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.563814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b29c7-0961-444e-a7a2-ff1bb6c3361f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.634013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.634680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.634993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.635266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.635989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.637719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fac923-49b1-458e-980d-3e6bbe9dc5f2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.647481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1113/2226] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:53.651583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.661796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fc8c5-c525-4f8e-94ce-47867dbdb23b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.667232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.668209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.668509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.668824 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] PATH_INFO: `/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b/users/9f82cc9175b94fa68ebfcc6a0fabf5a3/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.675562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b8df2a2edd4fd9847caf73ce44461b, user_id=9f82cc9175b94fa68ebfcc6a0fabf5a3, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.678331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.690112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1114/2227] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:53 2022] PUT /identity/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b/users/9f82cc9175b94fa68ebfcc6a0fabf5a3/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:53.701646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.701974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.702256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.729815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] MFA Rules not processed for user `9f82cc9175b94fa68ebfcc6a0fabf5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:53.730975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:53.804740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1114/2228] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:02:53.816295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:53.818041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:53.878521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bafc6-493a-42c7-8727-b6c348482cfc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:53.882131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:53.882771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:53.883049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:53.883327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:53.892320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:53.893869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6869-9562-4e1e-98bc-bea093784f19 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:53.903963 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1115/2229] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:54.033174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:54.097541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccc1fe-471e-4f11-8a78-8c8fd26f6cfc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:54.109469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:54.110371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:54.110654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:54.110932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] PATH_INFO: `/v3/users/9798e7e922b347e58fc118eaa788154b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:54.112201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9798e7e922b347e58fc118eaa788154b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:54.114189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:54.119786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:54.165786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9798e7e922b347e58fc118eaa788154b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.174419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9798e7e922b347e58fc118eaa788154b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.174931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9798e7e922b347e58fc118eaa788154b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.175552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9798e7e922b347e58fc118eaa788154b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.188015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9798e7e922b347e58fc118eaa788154b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.203455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1115/2230] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:54 2022] DELETE /identity/v3/users/9798e7e922b347e58fc118eaa788154b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:54.208525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:54.208845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:54.209132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:54.232476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:54.233659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:54.246451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:54.295435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1116/2231] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:54.305206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:54.306665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:54.370069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb6556-6cb6-44c4-b4d4-21d0edc0f9a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:54.373780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:54.374458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:54.374745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:54.375038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:54.383836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:54.386044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:54.395387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1116/2232] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:54.690018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:54.691449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:54.704941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511b8e4-ed31-49d6-a8a1-ce9cf537f0e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:54.709500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:54.710373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:54.710659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:54.710936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] PATH_INFO: `/v3/projects/f5f6ff20978445c28a479e8a08eda113` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:54.712125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f6ff20978445c28a479e8a08eda113)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:54.714322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:54.765477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f6ff20978445c28a479e8a08eda113', 'request_id': 'req-81d0e6f3-566e-48de-8047-df52ccf48e17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.771601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f6ff20978445c28a479e8a08eda113', 'request_id': 'req-81d0e6f3-566e-48de-8047-df52ccf48e17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.779273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f6ff20978445c28a479e8a08eda113', 'request_id': 'req-81d0e6f3-566e-48de-8047-df52ccf48e17'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.789648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] The token cache is being invalidate because project f5f6ff20978445c28a479e8a08eda113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:54.789937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.790560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.792327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1117/2233] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:54 2022] DELETE /identity/v3/projects/f5f6ff20978445c28a479e8a08eda113 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:54.796315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:54.808409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af21a9-43ee-4bef-803a-c119ab8a7b59 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:54.880844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:54.881883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:54.882290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:54.882683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] PATH_INFO: `/v3/users/9f82cc9175b94fa68ebfcc6a0fabf5a3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:54.884128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f82cc9175b94fa68ebfcc6a0fabf5a3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:54.886670 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:54.918519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:54.947025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f82cc9175b94fa68ebfcc6a0fabf5a3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.954627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f82cc9175b94fa68ebfcc6a0fabf5a3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.955380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f82cc9175b94fa68ebfcc6a0fabf5a3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.956081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f82cc9175b94fa68ebfcc6a0fabf5a3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.969551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b1590eb8-89ea-421d-9516-7c53a6b67c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f82cc9175b94fa68ebfcc6a0fabf5a3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:54.984398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1117/2234] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:02:54 2022] DELETE /identity/v3/users/9f82cc9175b94fa68ebfcc6a0fabf5a3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:55.289561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.291804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:55.308366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e6f3-566e-48de-8047-df52ccf48e17 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.366959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.367825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.368109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.368387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] PATH_INFO: `/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.369635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b8df2a2edd4fd9847caf73ce44461b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.371530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.416780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:55.420218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b8df2a2edd4fd9847caf73ce44461b', 'request_id': 'req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:55.427488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b8df2a2edd4fd9847caf73ce44461b', 'request_id': 'req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:55.435155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b8df2a2edd4fd9847caf73ce44461b', 'request_id': 'req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:55.445635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] The token cache is being invalidate because project d2b8df2a2edd4fd9847caf73ce44461b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:02:55.445921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:55.446547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22d77ebd-f7eb-4ea4-b6f8-78571666fb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:02:55.448070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1118/2235] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:02:55 2022] DELETE /identity/v3/projects/d2b8df2a2edd4fd9847caf73ce44461b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:55.453312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.453673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.454083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.472234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:55.473477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:55.532772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1118/2236] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:55.540277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.540576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.540892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.559417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:55.560637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:55.568507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1119/2237] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:55.572645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.582203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911eff8-b6ae-4c9d-8cd3-74f93d2fb815 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.586747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.587387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.587694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.588486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.589384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:02:55.590836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.592461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.605375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1119/2238] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:02:55.609583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.618685 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e8934-f292-4176-a84a-bd94fe5fa3e2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.623842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.624533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.625019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.625277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.625987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.627884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.658451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1120/2239] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:55 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:02:55.659895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.674142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fef91-861c-4e4c-a921-86511c89f60b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.741514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.742278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.742636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.742922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.743628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.745691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.770794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1120/2240] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:55 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:55.771807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.783769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a8c6-abfe-4eff-bfd1-74429106e473 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.788405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.789093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.789383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.789661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.790392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.793088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.803790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1121/2241] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:55.809535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.820213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89e228-9ffa-4ddc-82cf-a7d5c558e79f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.825168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.825831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.826120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.826400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.827198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.828839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab92ad-3e1a-4dd8-af3f-179289e0a6ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.838798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1121/2242] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:55.842993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:55.853887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a12c8-a0ed-446a-901b-1a6376256cfa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:55.858726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:55.859598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.859889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.860172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] PATH_INFO: `/v3/projects/c503192e18e24145967f60f344f4b9ff/users/e41bd08a943f465398edb09b86c34a43/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.875716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c503192e18e24145967f60f344f4b9ff, user_id=e41bd08a943f465398edb09b86c34a43, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:55.878433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:55.892252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1122/2243] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:55 2022] PUT /identity/v3/projects/c503192e18e24145967f60f344f4b9ff/users/e41bd08a943f465398edb09b86c34a43/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:55.903349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:55.903737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:55.904021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:55.924214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:55.933553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] MFA Rules not processed for user `e41bd08a943f465398edb09b86c34a43`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:55.935386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:55.999866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1122/2244] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:56.004426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.068039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628e1e-8c12-49a3-ae81-1cd0bf7c49ef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.073126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.073778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.074066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.074351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.075090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.077262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.103333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1123/2245] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:02:56 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:56.109071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.120132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dc0f3-1439-47d1-8e3e-42d6dd32ba88 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.176945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.177616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.177901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.178190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.178866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.180829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.201283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1123/2246] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:02:56 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:02:56.205884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.218147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73ee3-6ec0-4fe2-97a8-1cb1d03922d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.222734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.223440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.223732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.224013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.224800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.226687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.240140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1124/2247] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:02:56.244553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.255869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4d02-5065-4ba5-bbec-3f3d8d08f052 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.261326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.262195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.262512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.262787 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] PATH_INFO: `/v3/projects/4f2e6a90fb484e41a7e8c439064360c1/users/11b7a70ba62648338dda99e3432cfc30/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.278421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2e6a90fb484e41a7e8c439064360c1, user_id=11b7a70ba62648338dda99e3432cfc30, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.281124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.294753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1124/2248] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:56 2022] PUT /identity/v3/projects/4f2e6a90fb484e41a7e8c439064360c1/users/11b7a70ba62648338dda99e3432cfc30/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:56.298918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.309521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63c42-9df4-46db-b0f6-c3e988cff8d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.356183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:56.374441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.375481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.375942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.376398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.377751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.386623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e559ab-9679-4e8b-89eb-261a44e96006 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.389893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1125/2249] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:02:56.393957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.405139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1b5a5-71a9-4a63-8039-8f2379dc48d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.409816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.410753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.411065 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.411372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] PATH_INFO: `/v3/projects/4f2e6a90fb484e41a7e8c439064360c1/users/11b7a70ba62648338dda99e3432cfc30/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.413496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2e6a90fb484e41a7e8c439064360c1, user_id=11b7a70ba62648338dda99e3432cfc30, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.416875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.427007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1125/2250] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:02:56 2022] PUT /identity/v3/projects/4f2e6a90fb484e41a7e8c439064360c1/users/11b7a70ba62648338dda99e3432cfc30/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:02:56.437584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.437876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.438217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.465994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] MFA Rules not processed for user `11b7a70ba62648338dda99e3432cfc30`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:02:56.467206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:56.534498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1126/2251] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:02:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:02:56.545167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.546958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:02:56.604482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b21660-59d7-4043-b378-b80bb7f8573b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.608143 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.608841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.609138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.609423 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.616274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:02:56.618835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.620739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.630973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1126/2252] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:02:56.797909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:02:56.809081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efa619-fd90-4aad-81cd-9f55712629c7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:02:56.817527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:02:56.818193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:02:56.818481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:02:56.818787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:02:56.879081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:02:56.880521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaa43a-18bb-4c00-9349-64d9979c21bd service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:02:56.890612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1127/2253] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:01.634128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:01.635330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:01.701106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7159d-1c20-4de9-b058-1edc337718a7 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:01.705700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:01.706505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:01.706785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:01.707066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] PATH_INFO: `/v3/users/c8b0b4f1021c4663a9ffd1621402dc32` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:01.708224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b0b4f1021c4663a9ffd1621402dc32)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:01.710195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:01.768736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b0b4f1021c4663a9ffd1621402dc32'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:01.775963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b0b4f1021c4663a9ffd1621402dc32'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:01.776735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b0b4f1021c4663a9ffd1621402dc32'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:01.777405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b0b4f1021c4663a9ffd1621402dc32'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:01.789377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b0b4f1021c4663a9ffd1621402dc32'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:01.803457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1127/2254] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:01 2022] DELETE /identity/v3/users/c8b0b4f1021c4663a9ffd1621402dc32 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:01.809218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:01.809891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:01.809891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:01.827456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:01.828712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:01.894328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1128/2255] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:01.904502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:01.905966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:01.969521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38dff-078e-4c1d-8ee7-b717996326da admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:01.973198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:01.973846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:01.974129 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:01.974404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:01.985265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:01.986709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:01.997761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1128/2256] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:02.290293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:02.293007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:02.302420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:02.311862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8472a-dca9-4ed5-a965-14633fdf61a8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:02.317663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:02.318557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:02.318878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:02.319177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] PATH_INFO: `/v3/projects/64a7092266f54fc68a03430b81f45b12` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:02.320361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a7092266f54fc68a03430b81f45b12)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:02.322359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:02.381874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a7092266f54fc68a03430b81f45b12', 'request_id': 'req-540d5822-0cb0-4431-82fb-575c5423480c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.388821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a7092266f54fc68a03430b81f45b12', 'request_id': 'req-540d5822-0cb0-4431-82fb-575c5423480c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.396755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a7092266f54fc68a03430b81f45b12', 'request_id': 'req-540d5822-0cb0-4431-82fb-575c5423480c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.409829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] The token cache is being invalidate because project 64a7092266f54fc68a03430b81f45b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:02.410416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.411318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.413710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1129/2257] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:02 2022] DELETE /identity/v3/projects/64a7092266f54fc68a03430b81f45b12 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:02.418620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:02.431293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7690-9801-4247-b376-02b3dcceb68a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:02.459112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:02.514813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:02.516457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:02.517022 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:02.517134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] PATH_INFO: `/v3/users/7fd0942062b84621b4f6eb66992f3229` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:02.518249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd0942062b84621b4f6eb66992f3229)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:02.520184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:02.570246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd0942062b84621b4f6eb66992f3229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.577416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd0942062b84621b4f6eb66992f3229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.578082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd0942062b84621b4f6eb66992f3229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.578678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd0942062b84621b4f6eb66992f3229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.593091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd0942062b84621b4f6eb66992f3229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:02.607595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1129/2258] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:02 2022] DELETE /identity/v3/users/7fd0942062b84621b4f6eb66992f3229 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:02.913385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:02.914981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:02.928986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5822-0cb0-4431-82fb-575c5423480c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:02.987114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:02.987911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:02.988205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:02.988554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] PATH_INFO: `/v3/projects/aba64a0528904fcc80227eae89cd2600` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:02.989857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba64a0528904fcc80227eae89cd2600)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:02.991848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.039904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba64a0528904fcc80227eae89cd2600', 'request_id': 'req-e8c5d900-5083-4698-b06b-651af549eac7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.047012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba64a0528904fcc80227eae89cd2600', 'request_id': 'req-e8c5d900-5083-4698-b06b-651af549eac7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.053993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba64a0528904fcc80227eae89cd2600', 'request_id': 'req-e8c5d900-5083-4698-b06b-651af549eac7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.065437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] The token cache is being invalidate because project aba64a0528904fcc80227eae89cd2600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:03.065732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.066700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.071139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1130/2259] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:02 2022] DELETE /identity/v3/projects/aba64a0528904fcc80227eae89cd2600 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:03.072280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:03.073918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:03.089270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285b6a-bca7-459e-bb41-31987d126f61 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:03.150723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:03.152123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.152401 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.152702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] PATH_INFO: `/v3/users/54b404d7459e4ddfb897ce4faea57656` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.154009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b404d7459e4ddfb897ce4faea57656)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:03.155929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.195629 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:03.203086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b404d7459e4ddfb897ce4faea57656'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.210733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b404d7459e4ddfb897ce4faea57656'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.211397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b404d7459e4ddfb897ce4faea57656'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.212015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b404d7459e4ddfb897ce4faea57656'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.224188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b0de7267-e487-45da-b673-0ac79131347d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b404d7459e4ddfb897ce4faea57656'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.236551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1130/2260] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:03 2022] DELETE /identity/v3/users/54b404d7459e4ddfb897ce4faea57656 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:03.509495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:03.511676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:03.534802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d900-5083-4698-b06b-651af549eac7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:03.549086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:03.595730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:03.596591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.596910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.597190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] PATH_INFO: `/v3/projects/624c74557d2848d68847e844b67c3a9a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.598373 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=624c74557d2848d68847e844b67c3a9a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:03.600517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.647275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624c74557d2848d68847e844b67c3a9a', 'request_id': 'req-883ec2ba-b5a3-456b-802f-46b9b7347e4a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.654315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624c74557d2848d68847e844b67c3a9a', 'request_id': 'req-883ec2ba-b5a3-456b-802f-46b9b7347e4a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.662665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624c74557d2848d68847e844b67c3a9a', 'request_id': 'req-883ec2ba-b5a3-456b-802f-46b9b7347e4a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.673304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] The token cache is being invalidate because project 624c74557d2848d68847e844b67c3a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:03.673598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.674222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-883ec2ba-b5a3-456b-802f-46b9b7347e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:03.675757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1131/2261] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:03 2022] DELETE /identity/v3/projects/624c74557d2848d68847e844b67c3a9a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:03.681811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.682620 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.682930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.700568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:03.701782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:03.760840 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1131/2262] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:03.769512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.769826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.770144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.788206 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:03.789420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:03.797653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1132/2263] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:03.801920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:03.813404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73548875-42b4-4bf8-9d8a-a3eec8843043 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:03.817987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:03.818645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.818929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.819204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.819936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:03.820242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:03.821897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.835041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1132/2264] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:03.839205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:03.849907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8898d1-79e3-45e5-b24a-9f4f3b96f96f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:03.855296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:03.859741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.860041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.860440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.861298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:03.864910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.901713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1133/2265] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:03 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:03.902909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:03.913377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94ba6-f73a-4e36-a42b-9328b65e9b95 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:03.971776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:03.972453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:03.972802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:03.973078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:03.973936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:03.976009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:03.998846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1133/2266] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:03 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:04.002804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.013559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2afe3-72da-4d63-af3a-083d9788ab4b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.023797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.024520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.024842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.025122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.025903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.027752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.046770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1134/2267] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:04.048254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.059624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9a79-96eb-4500-aef2-364d36ff62a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.064210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.064919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.065209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.065486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.066215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.067824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862baa21-a8eb-45ea-bcb0-2f7b9431b61c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.077860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1134/2268] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:04.081952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.091990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebde73-4b5d-4ff0-b475-c8d21d7fdbeb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.096610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.097504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.097796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.098078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] PATH_INFO: `/v3/projects/3b6844813a2d4f9cbe0b5d9cd978b94e/users/259424a02e204265be6482b4537a3218/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.114100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6844813a2d4f9cbe0b5d9cd978b94e, user_id=259424a02e204265be6482b4537a3218, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.116805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.137858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1135/2269] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:04 2022] PUT /identity/v3/projects/3b6844813a2d4f9cbe0b5d9cd978b94e/users/259424a02e204265be6482b4537a3218/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:04.150175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.150469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.150807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.184521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] MFA Rules not processed for user `259424a02e204265be6482b4537a3218`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:04.186007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:04.196453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:04.255493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1135/2270] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:04.260076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.311940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:04.332742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021316a5-7e66-445b-88bb-c989efaafdb3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.337399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.338055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.338929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.339335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.340165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.342100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.367416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1136/2271] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:04 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:04.373166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.383656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08248a4f-b6e2-48c7-8160-b4d9ac0f8076 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.439554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.440225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.440510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.440818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.441522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.443466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.462824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1136/2272] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:04 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:04.467410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.478066 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e532-1e23-410f-9318-e715096b2660 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.483483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.484142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.484433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.484753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.485495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.487094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.498011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1137/2273] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:04.502103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.513213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4193f-c0f9-48b7-a199-713a7e37cc48 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.519495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.519495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.519495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.519829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] PATH_INFO: `/v3/projects/7d5092180e2d4437b333352b6e78df6a/users/537bf088ae3e432899ce6c9e2a62f82a/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.535960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5092180e2d4437b333352b6e78df6a, user_id=537bf088ae3e432899ce6c9e2a62f82a, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.538841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.556726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1137/2274] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:04 2022] PUT /identity/v3/projects/7d5092180e2d4437b333352b6e78df6a/users/537bf088ae3e432899ce6c9e2a62f82a/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:04.561392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.572477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c062d31-ba2d-467e-85f9-9febf4042528 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.644246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.645256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.645821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.646242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.647389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.649687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb300a1-b04e-449f-9db7-d1711a4d329f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.667177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1138/2275] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:04.673913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.687701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c84-659b-42a2-b681-cfee27913720 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.692442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.693352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.693635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.693912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] PATH_INFO: `/v3/projects/7d5092180e2d4437b333352b6e78df6a/users/537bf088ae3e432899ce6c9e2a62f82a/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.695870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5092180e2d4437b333352b6e78df6a, user_id=537bf088ae3e432899ce6c9e2a62f82a, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.699057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.709806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1138/2276] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:04 2022] PUT /identity/v3/projects/7d5092180e2d4437b333352b6e78df6a/users/537bf088ae3e432899ce6c9e2a62f82a/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:04.721463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.722358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.722731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.752567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] MFA Rules not processed for user `537bf088ae3e432899ce6c9e2a62f82a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:04.754677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:04.827356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1139/2277] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:04.839798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:04.841624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:04.902573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54cda-8f78-4ad2-bed1-281d71b28032 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:04.906562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:04.907280 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:04.907633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:04.907985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:04.919140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:04.920595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:04.933256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1139/2278] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:05.009034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:05.020169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3b7ff-ab65-4959-af66-357fc929c659 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:05.024284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:05.025527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:05.026566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:05.027003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:05.072134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:05.096597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:05.098127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567bd94-a852-4ab0-a6e5-344df7c53254 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:05.109248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1140/2279] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:05.621201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:05.633086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:05.697397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de280-09cc-4670-a2d6-364540d180eb service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:05.702392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:05.703264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:05.703545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:05.703826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] PATH_INFO: `/v3/users/259424a02e204265be6482b4537a3218` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:05.705054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=259424a02e204265be6482b4537a3218)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:05.707012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:05.755484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259424a02e204265be6482b4537a3218'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:05.762416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259424a02e204265be6482b4537a3218'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:05.763084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259424a02e204265be6482b4537a3218'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:05.763684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259424a02e204265be6482b4537a3218'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:05.775334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259424a02e204265be6482b4537a3218'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:05.788810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1140/2280] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:05 2022] DELETE /identity/v3/users/259424a02e204265be6482b4537a3218 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:05.794178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:05.794473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:05.794756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:05.812823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:05.814047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:05.874554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1141/2281] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:05.884130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:05.885694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:05.945621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6f2db-c29e-43b6-8924-35f120613d26 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:05.949557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:05.950242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:05.950533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:05.950810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:05.960827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:05.962338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:05.974059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1141/2282] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:06.271362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:06.273010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:06.288977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7fb0c-e7f3-40c1-822e-3d955bcd5089 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:06.294854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:06.296143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:06.296656 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:06.297410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] PATH_INFO: `/v3/projects/3b6844813a2d4f9cbe0b5d9cd978b94e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:06.299883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6844813a2d4f9cbe0b5d9cd978b94e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:06.303262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:06.364723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6844813a2d4f9cbe0b5d9cd978b94e', 'request_id': 'req-6923e83b-2544-405c-9144-22fdfa01f00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.373041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6844813a2d4f9cbe0b5d9cd978b94e', 'request_id': 'req-6923e83b-2544-405c-9144-22fdfa01f00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.381385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6844813a2d4f9cbe0b5d9cd978b94e', 'request_id': 'req-6923e83b-2544-405c-9144-22fdfa01f00c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.398607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] The token cache is being invalidate because project 3b6844813a2d4f9cbe0b5d9cd978b94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:06.399042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.399760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.402635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1142/2283] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:06 2022] DELETE /identity/v3/projects/3b6844813a2d4f9cbe0b5d9cd978b94e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:06.406439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:06.420376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765363-46c5-440e-a399-76db4c3de17f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:06.455012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:06.497689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:06.498581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:06.498873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:06.499152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] PATH_INFO: `/v3/users/537bf088ae3e432899ce6c9e2a62f82a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:06.500604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537bf088ae3e432899ce6c9e2a62f82a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:06.502565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:06.559246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537bf088ae3e432899ce6c9e2a62f82a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.570201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537bf088ae3e432899ce6c9e2a62f82a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.570701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537bf088ae3e432899ce6c9e2a62f82a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.572420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537bf088ae3e432899ce6c9e2a62f82a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.590771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3cbccfec-44bd-404b-a93c-1d05be9f6428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537bf088ae3e432899ce6c9e2a62f82a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:06.617426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1142/2284] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:06 2022] DELETE /identity/v3/users/537bf088ae3e432899ce6c9e2a62f82a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:06.919431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:06.921088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:06.937377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e83b-2544-405c-9144-22fdfa01f00c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.021301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.022180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.022465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.022744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] PATH_INFO: `/v3/projects/7d5092180e2d4437b333352b6e78df6a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.024078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5092180e2d4437b333352b6e78df6a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.026311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.029350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:07.087256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5092180e2d4437b333352b6e78df6a', 'request_id': 'req-9fb1df04-532c-480e-a63c-fd72c68f65f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.095432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5092180e2d4437b333352b6e78df6a', 'request_id': 'req-9fb1df04-532c-480e-a63c-fd72c68f65f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.103394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5092180e2d4437b333352b6e78df6a', 'request_id': 'req-9fb1df04-532c-480e-a63c-fd72c68f65f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.114900 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] The token cache is being invalidate because project 7d5092180e2d4437b333352b6e78df6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:07.115178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.115954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9fb1df04-532c-480e-a63c-fd72c68f65f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.117561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1143/2285] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:06 2022] DELETE /identity/v3/projects/7d5092180e2d4437b333352b6e78df6a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:07.123506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.123974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.124059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.142354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:07.143557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.208133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1143/2286] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:07.217286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.217581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.217870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.236783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:07.238001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.246854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1144/2287] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:07.251068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.261003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80012ef9-cbac-4568-97a3-2d20cf7ed0ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.265740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.266480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.266766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.267049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.267808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:07.268158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.269864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.282895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1144/2288] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:07.287279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.297298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21595d-1bc4-4154-a0ab-86d56459e272 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.302220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.302875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.303163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.303442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.304152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.306090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.336583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1145/2289] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:07.337725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.347449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.349205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.350589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e9e7c-2c10-43f0-943a-3ab08a77ebbe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.384626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:07.411533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.412207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.412492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.412804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.413512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.415464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.418254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a09c-f952-43ca-8085-c91779a2ae31 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.422965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.424050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.424190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.424478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] PATH_INFO: `/v3/users/e41bd08a943f465398edb09b86c34a43` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.425847 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41bd08a943f465398edb09b86c34a43)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.427921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.435209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1145/2290] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:07.439283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.448839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a673b-387a-4630-9b15-730deea74920 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.453558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.454179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.454469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.454746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.455490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.457167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.468605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1146/2291] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:07.472299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.482063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41bd08a943f465398edb09b86c34a43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.482947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64703f-2ac8-46b3-a696-eb5c4554b180 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.487853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.488589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.488958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.489242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.489906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41bd08a943f465398edb09b86c34a43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.490037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.490735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41bd08a943f465398edb09b86c34a43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.491677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.498349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41bd08a943f465398edb09b86c34a43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.500364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:07.502736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1147/2292] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:07.506961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.515018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0db609ae-2e2a-47b6-93f7-316bb6d0b274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41bd08a943f465398edb09b86c34a43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.517041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e40e-2fe1-4c35-886f-9a8fe62dcb34 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.521566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1146/2293] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:07 2022] DELETE /identity/v3/users/e41bd08a943f465398edb09b86c34a43 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:07.521995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.522931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.523221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.523501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] PATH_INFO: `/v3/projects/8baeef954dd9469fb906af4017375dff/users/2c61943bdc5442878c001aae0677adff/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.527313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.528683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.529053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.539495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8baeef954dd9469fb906af4017375dff, user_id=2c61943bdc5442878c001aae0677adff, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.542956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b58c35-31a9-48c6-8e6c-6fe516cf1539 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.547848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:07.549230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.557345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1148/2294] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:07 2022] PUT /identity/v3/projects/8baeef954dd9469fb906af4017375dff/users/2c61943bdc5442878c001aae0677adff/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:07.562072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.562366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.562651 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.579361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:07.580569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.616110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1147/2295] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:07.623310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.624683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.641681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1149/2296] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:07.654152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.655877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.688559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dce89d-1726-420b-a556-b32ef7a9eb92 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.732171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a57831-d0d5-4d11-8a10-2a444aaabe3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.735947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.736766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.737343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.737604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.748191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.748407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.749095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.750636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.750636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.750854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.761495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1150/2297] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:07.762064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.763577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98a900-1e0f-48df-9b6d-d546c0a18303 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.775429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1148/2298] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:07.857961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:07.859319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:07.873081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57596a-4c9f-442c-ae64-41ff6aa6badf service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:07.878721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:07.879748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.880126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.880498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] PATH_INFO: `/v3/users/314a24145bc448e0b23dfd88968b46bf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:07.881945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=314a24145bc448e0b23dfd88968b46bf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:07.884025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:07.904012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:07.938805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314a24145bc448e0b23dfd88968b46bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.946064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314a24145bc448e0b23dfd88968b46bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.946802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314a24145bc448e0b23dfd88968b46bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.947472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314a24145bc448e0b23dfd88968b46bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.961480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314a24145bc448e0b23dfd88968b46bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:07.977100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1151/2299] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:07 2022] DELETE /identity/v3/users/314a24145bc448e0b23dfd88968b46bf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:07.982284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:07.982618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:07.982898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:08.004801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:08.006063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:08.086146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:08.120982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1149/2300] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:08.138558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:08.140130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:08.238799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:08.240529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:08.256133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb615c-3757-4be6-b1b7-51abdf1b0e72 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:08.261839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:08.262099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88d12-0d2e-4858-8f02-5a9b0192bd24 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:08.262832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:08.263399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:08.263784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:08.270748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:08.271627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:08.271964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:08.272854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] PATH_INFO: `/v3/projects/c503192e18e24145967f60f344f4b9ff` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:08.274311 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c503192e18e24145967f60f344f4b9ff)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:08.276867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:08.283021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:08.286127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:08.299965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1152/2301] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:08.353132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c503192e18e24145967f60f344f4b9ff', 'request_id': 'req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.364742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c503192e18e24145967f60f344f4b9ff', 'request_id': 'req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.381688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c503192e18e24145967f60f344f4b9ff', 'request_id': 'req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.404761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] The token cache is being invalidate because project c503192e18e24145967f60f344f4b9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:08.405212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.406404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.409407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1150/2302] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:08 2022] DELETE /identity/v3/projects/c503192e18e24145967f60f344f4b9ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:08.413646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:08.432663 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e170e-06ac-45ef-8530-f3d78dcf4e33 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:08.512214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:08.513938 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:08.514225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:08.514500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] PATH_INFO: `/v3/users/11b7a70ba62648338dda99e3432cfc30` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:08.515832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b7a70ba62648338dda99e3432cfc30)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:08.518467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:08.569215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:08.578288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b7a70ba62648338dda99e3432cfc30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.589610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b7a70ba62648338dda99e3432cfc30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.590358 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b7a70ba62648338dda99e3432cfc30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.590992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b7a70ba62648338dda99e3432cfc30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.614112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b7a70ba62648338dda99e3432cfc30'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.631081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:08.631626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1153/2303] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:08 2022] DELETE /identity/v3/users/11b7a70ba62648338dda99e3432cfc30 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:08.633260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:08.657639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5a19-d423-4dc2-91ed-c9c3811a44f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:08.767860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:08.769509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:08.770043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:08.770536 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] PATH_INFO: `/v3/projects/028915500ec242cab30066e4717f02ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:08.772196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=028915500ec242cab30066e4717f02ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:08.774599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:08.870633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '028915500ec242cab30066e4717f02ef', 'request_id': 'req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.878803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '028915500ec242cab30066e4717f02ef', 'request_id': 'req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.904767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '028915500ec242cab30066e4717f02ef', 'request_id': 'req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.911886 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:08.925085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] The token cache is being invalidate because project 028915500ec242cab30066e4717f02ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:08.926008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.927211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:08.934187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1151/2304] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:08 2022] DELETE /identity/v3/projects/028915500ec242cab30066e4717f02ef => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:08.935764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:08.937764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:08.956159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b068541-0e9d-4ef1-a7d0-741435a266a4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.022589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:09.024397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.056299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.058535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.058825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.059107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] PATH_INFO: `/v3/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.060500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f95946a3eec4c4ea0e9b3a5f7bd6fc6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.062777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.145584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f95946a3eec4c4ea0e9b3a5f7bd6fc6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.154622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f95946a3eec4c4ea0e9b3a5f7bd6fc6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.155515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f95946a3eec4c4ea0e9b3a5f7bd6fc6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.156470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f95946a3eec4c4ea0e9b3a5f7bd6fc6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.157083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dbc45-c87b-47c0-9e24-f75f3d81fd7b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.163850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.165647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.166918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.167261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] PATH_INFO: `/v3/projects/4f2e6a90fb484e41a7e8c439064360c1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.168588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f2e6a90fb484e41a7e8c439064360c1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.171787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.179634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03544f06-eea7-4262-afab-ef3556c0bd78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f95946a3eec4c4ea0e9b3a5f7bd6fc6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.216530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1154/2305] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:08 2022] DELETE /identity/v3/users/9f95946a3eec4c4ea0e9b3a5f7bd6fc6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:09.284972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f2e6a90fb484e41a7e8c439064360c1', 'request_id': 'req-2843c69b-211d-4ad7-a7a0-e53473291a71'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.293649 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f2e6a90fb484e41a7e8c439064360c1', 'request_id': 'req-2843c69b-211d-4ad7-a7a0-e53473291a71'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.306569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f2e6a90fb484e41a7e8c439064360c1', 'request_id': 'req-2843c69b-211d-4ad7-a7a0-e53473291a71'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.321821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] The token cache is being invalidate because project 4f2e6a90fb484e41a7e8c439064360c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:09.322409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.324397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2843c69b-211d-4ad7-a7a0-e53473291a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.327180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1152/2306] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:09 2022] DELETE /identity/v3/projects/4f2e6a90fb484e41a7e8c439064360c1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:09.334460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.334753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.335085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.363008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:09.365086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.407405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:09.488008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1155/2307] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:09.497249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.497723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.498397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.544749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:09.548154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.562641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1153/2308] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:09.567078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:09.584452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b7d5-54d7-4eda-b39d-5678f1ead404 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.592079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.592079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.592347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.592591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.593485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:09.593799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.596302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.618120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1156/2309] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:03:09.624513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:09.636975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f00485-e451-4b2b-b54c-356319a2c451 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.644743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:09.646751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.649763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.650456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.651883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.651883 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.658059 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.662809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed88b2-6bc4-48cc-b9c8-e92c485159fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.670504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.671384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.671677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.671955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] PATH_INFO: `/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.673433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8394df60b8f45e687cdb60e9eea5b1f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.675012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.677210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.727890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1154/2310] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:09.737327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:09.788712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8394df60b8f45e687cdb60e9eea5b1f', 'request_id': 'req-bdf85229-a8d0-4d55-8641-7f89700cb94c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.796745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8394df60b8f45e687cdb60e9eea5b1f', 'request_id': 'req-bdf85229-a8d0-4d55-8641-7f89700cb94c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.805637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8394df60b8f45e687cdb60e9eea5b1f', 'request_id': 'req-bdf85229-a8d0-4d55-8641-7f89700cb94c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.826062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] The token cache is being invalidate because project d8394df60b8f45e687cdb60e9eea5b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:09.826062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.826950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bdf85229-a8d0-4d55-8641-7f89700cb94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:09.832549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1157/2311] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:09 2022] DELETE /identity/v3/projects/d8394df60b8f45e687cdb60e9eea5b1f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:09.833997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0607-3060-4207-98d6-e960db8c7f75 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.834309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0607-3060-4207-98d6-e960db8c7f75 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.834661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0607-3060-4207-98d6-e960db8c7f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.851965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:09.857169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26f9d-d532-499e-ba00-ab6185e86df2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:09.863647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e66b0607-3060-4207-98d6-e960db8c7f75 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:09.865493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e66b0607-3060-4207-98d6-e960db8c7f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.954686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1158/2312] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:09.962232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:09.962545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.962841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.963129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:09.963260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.964045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:09.964396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:09.965333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:09.967354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:09.983291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:09.984669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:09.999288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1159/2313] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:10.000038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1155/2314] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:09 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:10.000405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.001031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.002726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:10.012139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca0288-dd2f-4b19-ba87-a1d634c14659 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.015369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e335a-e16e-4d7a-af8c-e62a1d1204b8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.020256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.020978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.021266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.021546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.022315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.022441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.023241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.024051 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.025409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.026592 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.027533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:10.028067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.029923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.038228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1156/2315] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:10.042532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.048356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1160/2316] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:10.051786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.053576 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead35d3a-8e3b-4b5c-8d98-20e1a597541b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.059825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.060650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.060811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.061094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.061831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.063785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.067061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf72dce-a8b6-42f7-8d27-a637ceab9d90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.072175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.073466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.073882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.074163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.074867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.076858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.078095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1157/2317] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:10.082154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.093486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854181b-5f1c-4efa-95c6-c7dc58f79d1e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.098563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.100502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.100831 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.101129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] PATH_INFO: `/v3/projects/a689639fd49c4a4daee68586b7e050c4/users/4f18c52a63cc4be682e02a7dbee8a63a/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.117641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1161/2318] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:10.121710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.128699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a689639fd49c4a4daee68586b7e050c4, user_id=4f18c52a63cc4be682e02a7dbee8a63a, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.130174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:10.132546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fca0b-f491-47b6-a757-ad58e56317a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.156951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1158/2319] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:10 2022] PUT /identity/v3/projects/a689639fd49c4a4daee68586b7e050c4/users/4f18c52a63cc4be682e02a7dbee8a63a/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:10.164596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.165184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.165923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.202079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] MFA Rules not processed for user `4f18c52a63cc4be682e02a7dbee8a63a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:10.203747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:10.209453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096e04f-0572-49fb-ac66-1110c27ac705 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.282437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.283049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.283340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.283636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.284325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.286386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.318579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1162/2320] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:10.324389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1159/2321] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:10.328156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.336780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.337238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:10.346151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd08a-139c-43db-8687-266caa1387cb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.354555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.355788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.356282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.356807 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.357869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.360035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.374415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1163/2322] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:10.378060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.389559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:10.390676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9549-e0b7-48c1-b92e-54d89c338317 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.398469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.400728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.401654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.402148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.403105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.405166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.411762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46a39-4c89-42f7-b1d2-756f957834d6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.415746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.417554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.418204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.418509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.419591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1164/2323] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:10.425384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.431654 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.434156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5864f-5822-42fb-9836-5f6f34f892e9 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.437902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4bfa-f121-424b-9550-88b32674f2dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.442966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.443856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.444221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.444683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] PATH_INFO: `/v3/projects/e7520e3edc804cc3a78808f17461b766/users/05279ad713b449c5bded8452e25380da/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.446561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1160/2324] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:10.461762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7520e3edc804cc3a78808f17461b766, user_id=05279ad713b449c5bded8452e25380da, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.466354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.499430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1165/2325] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:10 2022] PUT /identity/v3/projects/e7520e3edc804cc3a78808f17461b766/users/05279ad713b449c5bded8452e25380da/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:10.501797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.502251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.502507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.540320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] MFA Rules not processed for user `05279ad713b449c5bded8452e25380da`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:10.542246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:10.644890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1161/2326] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:10.649346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.741075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:10.751771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d509269-c330-4ac7-968d-55fbadd6d662 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:10.759101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:10.761640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:10.762152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:10.762516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:10.763861 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:10.766145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:10.813127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1166/2327] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:10.814838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:10.833021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d19bff-6153-4716-8a30-d048c206fb50 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.027393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.028153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.028448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.028763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.039609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:11.042394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:11.120567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1162/2328] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:10 2022] POST /identity/v3/users => generated 299 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:11.122270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:11.135250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b651-ffdb-4179-a338-0dc0fb8e7fd3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.141599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.142422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.142796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.143140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.144020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:11.145854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:11.164717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1167/2329] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:11.166367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:11.181369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51f65b-25ae-443c-b8d0-04ee5cd6d036 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.186929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.187825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.188106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.188381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] PATH_INFO: `/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe/users/8035888bb9dd441a8f7de974db01aaa0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.217103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad75ddc79c54746a8e2ec0bbc7965fe, user_id=8035888bb9dd441a8f7de974db01aaa0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:11.221490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:11.256598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1163/2330] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:11 2022] PUT /identity/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe/users/8035888bb9dd441a8f7de974db01aaa0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:11.260393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:11.277681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07a3cb-eb36-4ffc-8f99-e4f446a7430f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.441773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.442470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.442749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.443023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.443784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:11.445516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333d7c5-4dce-4d36-951c-3e238b8470b8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:11.467463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1168/2331] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:11.473313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:11.507181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985983fb-50e9-4758-96f1-53baef0a6c92 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.534968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.536364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.536922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.538409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] PATH_INFO: `/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe/users/8035888bb9dd441a8f7de974db01aaa0/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.542283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad75ddc79c54746a8e2ec0bbc7965fe, user_id=8035888bb9dd441a8f7de974db01aaa0, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:11.545711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:11.559875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1164/2332] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:11 2022] PUT /identity/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe/users/8035888bb9dd441a8f7de974db01aaa0/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:11.570202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.570842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.571347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:11.613075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] MFA Rules not processed for user `8035888bb9dd441a8f7de974db01aaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:11.615451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:11.795099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1169/2333] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:11.811719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:11.814551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:11.913292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:11.988469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752db4-8e15-4724-a5f3-c6cb26718646 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:11.992981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:11.993653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:11.993944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:11.994230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:12.006489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:12.008856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430dd60-8ffd-41f0-96e9-630d6ec2b68c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:12.044820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1165/2334] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:12.084111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:12.242097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:12.261945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d590-f97b-414b-b4af-6399e86fc4b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:12.270727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:12.271750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:12.272852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:12.273246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] PATH_INFO: `/v3/users/05279ad713b449c5bded8452e25380da` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:12.277226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] RBAC: Authorizing `identity:delete_user(user_id=05279ad713b449c5bded8452e25380da)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:12.279689 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:12.384677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05279ad713b449c5bded8452e25380da'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:12.398550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05279ad713b449c5bded8452e25380da'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:12.399315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05279ad713b449c5bded8452e25380da'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:12.402163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05279ad713b449c5bded8452e25380da'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:12.427854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05279ad713b449c5bded8452e25380da'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:12.446188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1170/2335] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:12 2022] DELETE /identity/v3/users/05279ad713b449c5bded8452e25380da => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:12.452478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eb77-8bfd-419b-9e60-34b7679589ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:12.452800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eb77-8bfd-419b-9e60-34b7679589ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:12.453317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eb77-8bfd-419b-9e60-34b7679589ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:12.512933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-44b0eb77-8bfd-419b-9e60-34b7679589ff None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:12.516869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44b0eb77-8bfd-419b-9e60-34b7679589ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:12.630032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1166/2336] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:12.639542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:12.641384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:12.802562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3759f7-3252-4e07-b3e8-731b4a6b408b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:12.813934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:12.816723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:12.816723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:12.816723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:12.832561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:12.834325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:12.876920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1171/2337] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:13.465579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6739-928b-455d-ae0f-f82704987810 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.465977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6739-928b-455d-ae0f-f82704987810 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.466260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6739-928b-455d-ae0f-f82704987810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.500217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-380a6739-928b-455d-ae0f-f82704987810 None None] MFA Rules not processed for user `2c61943bdc5442878c001aae0677adff`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:13.501793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-380a6739-928b-455d-ae0f-f82704987810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:13.597547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:13.600928 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:13.608077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1167/2338] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:13.611527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6739-928b-455d-ae0f-f82704987810 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:13.616101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-380a6739-928b-455d-ae0f-f82704987810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:13.621083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183590-0344-4b96-84f8-b606c696f952 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:13.634439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6739-928b-455d-ae0f-f82704987810 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:13.634583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:13.637618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.638096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.639359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] PATH_INFO: `/v3/projects/e7520e3edc804cc3a78808f17461b766` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.640939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:13.641541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7520e3edc804cc3a78808f17461b766)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:13.641972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.642375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.642695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.643460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:13.644254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:13.645550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:13.689181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1168/2339] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:13 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:13.690816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:13.702559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7520e3edc804cc3a78808f17461b766', 'request_id': 'req-6faa0e63-fda8-4c13-9bd8-caa7509a4331'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.711100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7520e3edc804cc3a78808f17461b766', 'request_id': 'req-6faa0e63-fda8-4c13-9bd8-caa7509a4331'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.730209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7520e3edc804cc3a78808f17461b766', 'request_id': 'req-6faa0e63-fda8-4c13-9bd8-caa7509a4331'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.739932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:13.746535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:13.752434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] The token cache is being invalidate because project e7520e3edc804cc3a78808f17461b766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:13.752759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.753458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.756064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1172/2340] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:13 2022] DELETE /identity/v3/projects/e7520e3edc804cc3a78808f17461b766 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:13.757290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:13.761631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:13.784650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd0d06-780f-4a29-8570-9b6977ee51e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:13.854319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0e63-fda8-4c13-9bd8-caa7509a4331 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:13.859852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:13.860037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:13.860744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.860891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.861087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.861199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.861370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] PATH_INFO: `/v3/users/20c7cf1573374f36ad2d09b78ab968e8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.861620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.862432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:13.862607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c7cf1573374f36ad2d09b78ab968e8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:13.864569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:13.864971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:13.910332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1169/2341] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:13 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:13.911728 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:13.913555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:13.930661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabec9a-4b35-463b-b4aa-588af3eed158 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:13.937516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:13.938415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:13.938755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:13.938952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] PATH_INFO: `/v3/users/8035888bb9dd441a8f7de974db01aaa0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:13.940377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8035888bb9dd441a8f7de974db01aaa0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:13.943409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:13.958858 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c7cf1573374f36ad2d09b78ab968e8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.969392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c7cf1573374f36ad2d09b78ab968e8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.970307 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c7cf1573374f36ad2d09b78ab968e8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.971580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c7cf1573374f36ad2d09b78ab968e8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:13.996961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c7cf1573374f36ad2d09b78ab968e8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.007602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8035888bb9dd441a8f7de974db01aaa0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.023000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8035888bb9dd441a8f7de974db01aaa0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.023889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8035888bb9dd441a8f7de974db01aaa0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.027898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1173/2342] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:13 2022] DELETE /identity/v3/users/20c7cf1573374f36ad2d09b78ab968e8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:14.028929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.031412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:14.038064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8035888bb9dd441a8f7de974db01aaa0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.058162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c40cf07f-db21-48f1-9e8f-77179e6c21f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8035888bb9dd441a8f7de974db01aaa0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:14.073291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1170/2343] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:13 2022] DELETE /identity/v3/users/8035888bb9dd441a8f7de974db01aaa0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:14.077787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.078074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.078372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.110513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:14.110513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:14.130549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc60ccc-9c18-4e0e-a5b4-babab2706a6a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:14.172681 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:14.176199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:14.208760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1171/2344] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:14.221480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.222936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:14.237679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:14.238658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.239120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.239616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.240793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:14.242978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:14.269830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1174/2345] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:14.274158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.278857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:14.292556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fad0d-2cb2-4f9c-ae03-51957cc7cc78 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:14.298801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:14.299805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.300267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.300756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.314682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:14.318020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:14.322123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff710e-1af0-4bb9-96b9-558d0666837c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:14.354402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:14.355369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.355658 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.355942 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] PATH_INFO: `/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b/users/0a3bc2942c944831a81e59731beb244b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.357074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1172/2346] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:14.393263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45bdcf65b6a4a11a78979d6a23c2d6b, user_id=0a3bc2942c944831a81e59731beb244b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:14.395937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:14.428056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1175/2347] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:14 2022] PUT /identity/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b/users/0a3bc2942c944831a81e59731beb244b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:14.429321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.445357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051f5c8-41a8-42e8-af72-d7fa0ff1cc73 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:14.526323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:14.527180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.527480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.527765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.528524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:14.530308 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:14.550777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1173/2348] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:14.551831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.566336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd5bc-e434-405a-b2df-8d6f19e18e40 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:14.573654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:14.574896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:14.575252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:14.575605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] PATH_INFO: `/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b/users/0a3bc2942c944831a81e59731beb244b/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:14.580943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45bdcf65b6a4a11a78979d6a23c2d6b, user_id=0a3bc2942c944831a81e59731beb244b, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:14.584667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c158a-d21f-48bf-9d0b-ce29cd58a836 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:14.602759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1176/2349] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:14 2022] PUT /identity/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b/users/0a3bc2942c944831a81e59731beb244b/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:14.914678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:14.921766 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:14.935875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302472e-d124-4c9a-8d75-b6c51fdc6de7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.062947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.063908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.064243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.064532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] PATH_INFO: `/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.065959 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad75ddc79c54746a8e2ec0bbc7965fe)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.069166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.110510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:15.138746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad75ddc79c54746a8e2ec0bbc7965fe', 'request_id': 'req-75f23200-df94-4b8e-83d9-adb5df80a484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.145555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad75ddc79c54746a8e2ec0bbc7965fe', 'request_id': 'req-75f23200-df94-4b8e-83d9-adb5df80a484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.154436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad75ddc79c54746a8e2ec0bbc7965fe', 'request_id': 'req-75f23200-df94-4b8e-83d9-adb5df80a484'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.167023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] The token cache is being invalidate because project dad75ddc79c54746a8e2ec0bbc7965fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:15.167317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.168021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.169682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1174/2350] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:14 2022] DELETE /identity/v3/projects/dad75ddc79c54746a8e2ec0bbc7965fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:15.175467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27c88-be4e-4d03-b114-ada8f7389aaa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.175759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27c88-be4e-4d03-b114-ada8f7389aaa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.180755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27c88-be4e-4d03-b114-ada8f7389aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.207925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-abd27c88-be4e-4d03-b114-ada8f7389aaa None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:15.209300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abd27c88-be4e-4d03-b114-ada8f7389aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:15.262437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.264036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:15.320243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1177/2351] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:15.323977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.324392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.325054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.348476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:15.361655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:15.380203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1178/2352] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:15.386406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.402294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344a45-559e-43e1-817c-cbc7380f3b33 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.408936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.409976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.410440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.410882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.411768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f23200-df94-4b8e-83d9-adb5df80a484 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.411998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:15.412516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.416274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.421205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.422257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.422613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.422955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] PATH_INFO: `/v3/projects/4b09c15f97154b0b9cd5a9e80e9e54a6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.424420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b09c15f97154b0b9cd5a9e80e9e54a6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.426473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.438762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1179/2353] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:15.443560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.461378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93edb32-ffaf-483b-9730-19cdd1954875 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.497170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.498085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.498282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.498561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.499283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.503071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b09c15f97154b0b9cd5a9e80e9e54a6', 'request_id': 'req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.503607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.514015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b09c15f97154b0b9cd5a9e80e9e54a6', 'request_id': 'req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.534957 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b09c15f97154b0b9cd5a9e80e9e54a6', 'request_id': 'req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.552790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] The token cache is being invalidate because project 4b09c15f97154b0b9cd5a9e80e9e54a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:15.553104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.553776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:15.555435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1175/2354] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:15 2022] DELETE /identity/v3/projects/4b09c15f97154b0b9cd5a9e80e9e54a6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:15.585262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1180/2355] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:15 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:15.589420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.591406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:15.724450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:15.799107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10dee-8bae-40c8-9fc7-ee2a9ba4942d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.905079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.905988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.906275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.906557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.907428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.909713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.938624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1176/2356] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:15 2022] POST /identity/v3/users => generated 294 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:15.939570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.958378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32b2db-ba39-48a0-af76-6e70ab0e86cb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:15.964184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:15.965185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:15.965543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:15.965893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:15.966909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:15.968896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:15.980694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1181/2357] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:15.985475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:15.999523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fd9d2-6891-49ad-98ce-7c4787ac3b5a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.008203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.009064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.009478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.010473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.011330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.013401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e69870-e907-4082-ad4c-a0fbea912b13 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.058283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1177/2358] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:16.060014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.070704 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00719720-557f-4e39-bce1-5a0f87ab0a92 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.077581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.078478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.079502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.079800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] PATH_INFO: `/v3/projects/d660b772963548b9848c4dce33d1e867/users/b28d59189e1f4fd88026f5d0d73ff229/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.104004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d660b772963548b9848c4dce33d1e867, user_id=b28d59189e1f4fd88026f5d0d73ff229, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.108316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.132727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1182/2359] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:16 2022] PUT /identity/v3/projects/d660b772963548b9848c4dce33d1e867/users/b28d59189e1f4fd88026f5d0d73ff229/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:16.137166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.137510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.137799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.171627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] MFA Rules not processed for user `b28d59189e1f4fd88026f5d0d73ff229`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:16.171627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:16.252370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1178/2360] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:16.257106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.263844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:16.331470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19103b-d6ff-43d8-94b0-e598dc6d3d6c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.336211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.337013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.337372 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.337721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.338555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.340747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.382083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1183/2361] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:16 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:16.383462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.397158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b92ff-fdbc-4b3d-82f5-b7166d9da441 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.482055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:16.514726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.520078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.520480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.520803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.521533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.525544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.569586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1179/2362] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:16 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:16.574127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.586168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83549f8-4d53-4bac-88e5-00390cef253e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.596788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.597477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.597768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.598054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.598800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.600834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.621306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1184/2363] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:16.622539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.633574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1e6d-db17-4512-8689-4de55ce2e754 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.640717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.641972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.642082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.642328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] PATH_INFO: `/v3/projects/88c8630ea0d944bb837f48e1f4520011/users/bcaa0fb0210e40798715644eaced32c6/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.661819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c8630ea0d944bb837f48e1f4520011, user_id=bcaa0fb0210e40798715644eaced32c6, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.665067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.688826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1180/2364] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:16 2022] PUT /identity/v3/projects/88c8630ea0d944bb837f48e1f4520011/users/bcaa0fb0210e40798715644eaced32c6/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:16.690015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.703324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd3306-9955-4c41-a99b-2f85fab37582 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.790054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.792214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.792519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.792829 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.793819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.795691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbbbf9-0cb9-47ad-9ec0-68b2915b6a5f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.807315 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1185/2365] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:16.812202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:16.825230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc7e59-005a-4c44-8583-50f01b2e5ade admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:16.830117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:16.831064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.831353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.832320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] PATH_INFO: `/v3/projects/88c8630ea0d944bb837f48e1f4520011/users/bcaa0fb0210e40798715644eaced32c6/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.834932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c8630ea0d944bb837f48e1f4520011, user_id=bcaa0fb0210e40798715644eaced32c6, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:16.837735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:16.850952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1181/2366] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:16 2022] PUT /identity/v3/projects/88c8630ea0d944bb837f48e1f4520011/users/bcaa0fb0210e40798715644eaced32c6/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:16.863021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:16.864221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:16.864221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:16.903872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] MFA Rules not processed for user `bcaa0fb0210e40798715644eaced32c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:16.905776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:17.008541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:17.022776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1186/2367] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:17.031010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:17.033488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:17.142519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74320-e48a-4033-aa9e-8429b7b4db4a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:17.146452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:17.147319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:17.147489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:17.147871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:17.158499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:17.160011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cf0e0-533a-4373-9e2d-146746d799c6 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:17.171514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1182/2368] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:18.149823 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:18.296759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc950-4fe4-4247-a812-fbea88dc599d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:18.314605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:18.316857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:18.331019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:18.331316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] PATH_INFO: `/v3/users/b28d59189e1f4fd88026f5d0d73ff229` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:18.332801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b28d59189e1f4fd88026f5d0d73ff229)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:18.335655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:18.439299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b28d59189e1f4fd88026f5d0d73ff229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:18.455798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b28d59189e1f4fd88026f5d0d73ff229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:18.457586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b28d59189e1f4fd88026f5d0d73ff229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:18.458228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b28d59189e1f4fd88026f5d0d73ff229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:18.482780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b28d59189e1f4fd88026f5d0d73ff229'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:18.512422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1187/2369] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:18 2022] DELETE /identity/v3/users/b28d59189e1f4fd88026f5d0d73ff229 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:18.517289 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:18.517739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:18.517855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:18.552599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:18.555368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:18.881684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1183/2370] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:18.896457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:18.899465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:19.098218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca3d9-ce4b-4ae2-a5b6-cbb144420acd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:19.114441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:19.115377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:19.115699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:19.116153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:19.146685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:19.148458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:19.162451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1188/2371] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:20.526777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:20.528554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:20.564015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790876b4-cb7b-4715-aba4-b70291e370fc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:20.583129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:20.585006 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:20.585750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:20.586246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] PATH_INFO: `/v3/projects/d660b772963548b9848c4dce33d1e867` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:20.589264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d660b772963548b9848c4dce33d1e867)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:20.592413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:20.700762 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:20.708212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd660b772963548b9848c4dce33d1e867', 'request_id': 'req-fdcd243b-8993-4f19-83dc-22065c6aabfa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:20.719532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd660b772963548b9848c4dce33d1e867', 'request_id': 'req-fdcd243b-8993-4f19-83dc-22065c6aabfa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:20.735530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd660b772963548b9848c4dce33d1e867', 'request_id': 'req-fdcd243b-8993-4f19-83dc-22065c6aabfa'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:20.752370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] The token cache is being invalidate because project d660b772963548b9848c4dce33d1e867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:20.752739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:20.753399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:20.755300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1184/2372] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:20 2022] DELETE /identity/v3/projects/d660b772963548b9848c4dce33d1e867 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:20.759493 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:20.810666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d2eb-098b-492f-ba54-98e23a72d60e service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:20.845891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:20.923100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:20.924491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:20.925002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:20.925485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] PATH_INFO: `/v3/users/bcaa0fb0210e40798715644eaced32c6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:20.927084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcaa0fb0210e40798715644eaced32c6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:20.929465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:21.005365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcaa0fb0210e40798715644eaced32c6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.015290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcaa0fb0210e40798715644eaced32c6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.016366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcaa0fb0210e40798715644eaced32c6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.017258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcaa0fb0210e40798715644eaced32c6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.036243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-56e87ecb-e710-4d7e-9a72-a7aa7afbb6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcaa0fb0210e40798715644eaced32c6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.062102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1189/2373] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:20 2022] DELETE /identity/v3/users/bcaa0fb0210e40798715644eaced32c6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:21.526723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:21.528266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:21.562329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd243b-8993-4f19-83dc-22065c6aabfa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:21.618696 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa1fb6-736e-40e9-adec-05511455c611 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:21.619052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa1fb6-736e-40e9-adec-05511455c611 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:21.619360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa1fb6-736e-40e9-adec-05511455c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:21.656202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:21.657498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:21.658889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:21.660037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] PATH_INFO: `/v3/users/34bd2ff763fc4e12999884464df92e4e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:21.663298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bd2ff763fc4e12999884464df92e4e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:21.670757 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:21.715886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a9fa1fb6-736e-40e9-adec-05511455c611 None None] MFA Rules not processed for user `0a3bc2942c944831a81e59731beb244b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:21.717668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a9fa1fb6-736e-40e9-adec-05511455c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:21.764984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bd2ff763fc4e12999884464df92e4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.784251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bd2ff763fc4e12999884464df92e4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.785057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bd2ff763fc4e12999884464df92e4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.797653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bd2ff763fc4e12999884464df92e4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.859922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bd2ff763fc4e12999884464df92e4e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:21.874801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1185/2374] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:21 2022] DELETE /identity/v3/users/34bd2ff763fc4e12999884464df92e4e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:21.875782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:21.877408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:21.934164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1190/2375] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:21.941599 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:21.942052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:21.942512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:21.984074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:21.985337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.021376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:22.025170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:22.032013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26e6ec-08a0-4100-bcff-1fbcbdbf8571 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.038983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.039860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.040140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.042286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] PATH_INFO: `/v3/projects/88c8630ea0d944bb837f48e1f4520011` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.043724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c8630ea0d944bb837f48e1f4520011)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.046497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.073847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1191/2376] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:22.075126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.076692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.114468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c8630ea0d944bb837f48e1f4520011', 'request_id': 'req-51943e42-d3d3-4042-9faa-a3de0eae62f6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.122161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c8630ea0d944bb837f48e1f4520011', 'request_id': 'req-51943e42-d3d3-4042-9faa-a3de0eae62f6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.133545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c8630ea0d944bb837f48e1f4520011', 'request_id': 'req-51943e42-d3d3-4042-9faa-a3de0eae62f6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.146563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] The token cache is being invalidate because project 88c8630ea0d944bb837f48e1f4520011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:22.147003 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.147743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.149677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1186/2377] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:21 2022] DELETE /identity/v3/projects/88c8630ea0d944bb837f48e1f4520011 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:22.153709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.153709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.159435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded2b37-553d-4a72-aaad-15d8ae4b41d2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.218220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943e42-d3d3-4042-9faa-a3de0eae62f6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.222216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.222991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.223290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.223594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.225874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.240771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.241479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.241765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.242046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.243944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.320834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.321258 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.322579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139996c4-f4a5-4791-9040-5eeabfbe3f5c service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.323657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967c531-3f7e-49f4-8c62-5e0ada803647 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.336715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1187/2378] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:22.338653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.338653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.338934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.348951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1192/2379] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:22.358869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:22.360118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.369275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1188/2380] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:22.377138 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.377410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.377790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.402505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:22.404590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.437440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1193/2381] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:22.439915 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.451684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a909dc-ee5b-475c-b386-7320b1ed6bd5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.459743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.460506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.460868 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.461152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.461933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:22.462811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.464204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c091f2-80c0-4bed-a3ca-a57591c6d14a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.482689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1189/2382] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:22.487414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.500770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.501040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f6732-a208-4b13-b0dd-d3d344c2db1b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.501195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.501533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.507081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.507750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.508064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.508338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.513132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.513132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.528569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:22.530618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.544394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1194/2383] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:22.548235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.553278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:22.568316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:22.640768 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57236-3354-4dcc-95ca-161080732740 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.646119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.646811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.647097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.647391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.647695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1190/2384] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:22.649294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.652181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.696785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1195/2385] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:22.701607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.701793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.701894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.714832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c16da-d15d-485a-9010-d37c8a5fca03 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.720476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.721429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.721866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.725619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.725619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.725619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.748896 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1196/2386] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:22.750418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.758614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.772862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d9d9c-2ebe-466c-ba5f-ce7c21237556 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.778814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.779691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.779972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.780290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] PATH_INFO: `/v3/projects/735f29ca5dd9418e92da9b80a9373ac8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.781929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=735f29ca5dd9418e92da9b80a9373ac8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.783655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.802008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a2f2d-a114-43ac-bf20-3972b33dcffb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.834924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735f29ca5dd9418e92da9b80a9373ac8', 'request_id': 'req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.843291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735f29ca5dd9418e92da9b80a9373ac8', 'request_id': 'req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.851933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735f29ca5dd9418e92da9b80a9373ac8', 'request_id': 'req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.878374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.879193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.879574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.881068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] The token cache is being invalidate because project 735f29ca5dd9418e92da9b80a9373ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:22.881449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.882207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:22.884177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1197/2387] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:22 2022] DELETE /identity/v3/projects/735f29ca5dd9418e92da9b80a9373ac8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:22.885539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:22.887765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.887905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.890218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:22.960710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.962769 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00ff34-c72d-4aca-b7f1-ab0002223a12 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.973230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1191/2388] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:03:22.973425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7853d-6548-4ba3-ac97-eb2c88b146b9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:22.977227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d186-8374-4640-ad4d-26b6e2f9a348 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.977583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d186-8374-4640-ad4d-26b6e2f9a348 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.977937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d186-8374-4640-ad4d-26b6e2f9a348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.983765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:22.985850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:22.985997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:22.986353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:22.987242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:22.989552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:22.996690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:23.012588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a045d186-8374-4640-ad4d-26b6e2f9a348 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:23.013903 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a045d186-8374-4640-ad4d-26b6e2f9a348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:23.022793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1198/2389] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:23.037096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.039187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1192/2390] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:23.042004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.042324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.042587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.058350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea259d-3563-4ed5-a74f-5102aa08ddb6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.065241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.066412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.066803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.067181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] PATH_INFO: `/v3/projects/22c964de02c741139189c5d6b7c5d3e7/users/84d12f0fca5b4b10a0aba4b89461e58c/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.069909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:23.071539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:23.072039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:23.096107 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1193/2391] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:23.098298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.101519 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c964de02c741139189c5d6b7c5d3e7, user_id=84d12f0fca5b4b10a0aba4b89461e58c, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.106183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaddc2-8594-408e-b2b8-42b5471ff4c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.116168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76526-d7b4-4725-a24d-800c578be3c3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.130001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.130698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.130980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.131417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1199/2392] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:23 2022] PUT /identity/v3/projects/22c964de02c741139189c5d6b7c5d3e7/users/84d12f0fca5b4b10a0aba4b89461e58c/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:23.131540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.132326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:23.132754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.136887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.142367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.142871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.143244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.172832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1194/2393] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:23.174494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.207956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25124a-9ee6-4a1c-80aa-cf098aa4b234 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.224233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] MFA Rules not processed for user `84d12f0fca5b4b10a0aba4b89461e58c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:23.233573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:23.334967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.335843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.336184 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.336541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.337411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.339573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.389056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1195/2394] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:23 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:23.393124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.428923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1200/2395] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:23.441512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.443223 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:23.506835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af113ac3-5f43-4a46-b9fc-638e4ea469b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.513020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.513796 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.514153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.514501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.515386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.517632 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.535185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638eb95-3134-435a-ab79-679fe8f27623 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.552833 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1196/2396] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:23 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:23.560858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.574962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f28309-3280-4c84-8daa-bcc72bde35bb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.600710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.601427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.607081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.607465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.611573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.613284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.624188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.624988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.625334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.625688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.642930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1197/2397] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:23.647926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.661008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2073c7-d42f-4a8c-8059-4236d005a34f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.672178 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.673919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.674209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.674493 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.675528 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.677280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.685502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:23.705093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1198/2398] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:23.706300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.719717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.721681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.726416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3c10-6644-41af-8394-8a17dec898ab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.740346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.741725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.742097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.742439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] PATH_INFO: `/v3/projects/943870c3b29a4651885aa091a7c39a0e/users/44db050c06b84991b6a28be2aba39289/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.743139 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1201/2399] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:23.760496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943870c3b29a4651885aa091a7c39a0e, user_id=44db050c06b84991b6a28be2aba39289, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.763648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40790671-b234-47b0-b9f0-46a64f548b15 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.777198 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.794413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857da55b-934f-4214-a71c-a8122170d85a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:23.797223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1199/2400] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:23 2022] PUT /identity/v3/projects/943870c3b29a4651885aa091a7c39a0e/users/44db050c06b84991b6a28be2aba39289/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:23.802216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.802508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.802794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.829650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:23.841094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:23.849638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:23.892084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:23.893466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:23.893894 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:23.894579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] PATH_INFO: `/v3/users/84d12f0fca5b4b10a0aba4b89461e58c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:23.896464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d12f0fca5b4b10a0aba4b89461e58c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:23.898742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:23.961711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1200/2401] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:23.988991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:23.992207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:24.036318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d12f0fca5b4b10a0aba4b89461e58c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:24.052534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d12f0fca5b4b10a0aba4b89461e58c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:24.053632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d12f0fca5b4b10a0aba4b89461e58c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:24.054364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d12f0fca5b4b10a0aba4b89461e58c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:24.080794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-63af0e8e-ad9a-4684-b301-ff42b7ba7f66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d12f0fca5b4b10a0aba4b89461e58c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:24.098462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfc842-5dbc-4238-be8e-fb3f09757a29 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:24.104709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1202/2402] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:23 2022] DELETE /identity/v3/users/84d12f0fca5b4b10a0aba4b89461e58c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:24.106317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:24.106710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:24.107093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:24.144895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:24.146365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:24.203396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:24.203779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:24.204067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:24.204344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:24.206462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:24.270585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1203/2403] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:24.291309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:24.299023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:24.313589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:24.321895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea97fc-a443-4f22-aacd-43442596eed7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:24.326629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:24.328108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:24.331527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:24.332320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:24.332814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:24.333206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:24.342512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1201/2404] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:24.354512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:24.356282 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9e109-6514-4166-bad8-c4b52784062c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:24.383289 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1204/2405] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:24.937108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:24.939428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:24.958651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed620c-b1ce-44ca-9634-6aa5505da1e5 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:24.973339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:24.974227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:24.974511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:24.974789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] PATH_INFO: `/v3/projects/22c964de02c741139189c5d6b7c5d3e7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:24.977726 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c964de02c741139189c5d6b7c5d3e7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:24.983298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:25.007482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:25.093739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c964de02c741139189c5d6b7c5d3e7', 'request_id': 'req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:25.096237 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c964de02c741139189c5d6b7c5d3e7', 'request_id': 'req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:25.119750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c964de02c741139189c5d6b7c5d3e7', 'request_id': 'req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:25.178093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] The token cache is being invalidate because project 22c964de02c741139189c5d6b7c5d3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:25.178569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:25.179317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ac21e18-8e34-4fba-b317-ffb3d6d7aabb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:25.185559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1202/2406] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:24 2022] DELETE /identity/v3/projects/22c964de02c741139189c5d6b7c5d3e7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:25.186766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.187598 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.188030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.315164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:25.316971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:25.495684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.496718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.496718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.499631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1205/2407] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:25.519526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:25.522182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:25.539377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1203/2408] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:25.544986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:25.570079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af896b-afef-4f26-8d2a-7913a32fb471 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:25.585851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:25.589353 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.591747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.592275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.593302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:25.594906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:25.599031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db4e7a-d5c7-4a9b-a2f3-d35d850231d8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:25.633453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:25.646230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1206/2409] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:25.647468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2d66-f17d-4e64-8490-13d18e8f888d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:25.664293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:25.665425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.666524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.667232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.668343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:25.672718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:25.762391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.762642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1204/2410] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:25 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:03:25.766239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.767968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:25.768655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.833798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] MFA Rules not processed for user `44db050c06b84991b6a28be2aba39289`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:25.835998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:25.878155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373312-dcea-4fe0-bb54-3eaf3a852680 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:25.885086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:25.885976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.887154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.887449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.888609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:25.891054 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:25.940406 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1205/2411] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:25 2022] POST /identity/v3/users => generated 281 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:25.945515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:25.958842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:25.974804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf501f-b2fd-410e-b7e6-56f77b539112 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:25.980504 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1207/2412] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:25.989234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:25.990490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:25.990789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:25.991075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:25.996001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.008259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.011741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:26.013563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:26.058535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1206/2413] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:26.074851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:26.099363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad3b6-687d-4d77-906b-e2d051c2ce03 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:26.114922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:26.115501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.116351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.116475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.117934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.118973 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.150653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1207/2414] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:26.151962 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:26.189763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fac05-ae82-4118-8b00-961b95ad6c47 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:26.199460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:26.200682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.200976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.201262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] PATH_INFO: `/v3/projects/13fd41934fb346e2a9e578e87856518c/users/56f52f26f52242398fc932c049109af1/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.238551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13fd41934fb346e2a9e578e87856518c, user_id=56f52f26f52242398fc932c049109af1, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.239192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f156e30-f691-459d-a253-a9dc05ddde02 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:26.247346 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:26.248064 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.248958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.249254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.249539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.269867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.271440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e55877-7a6a-4a32-9512-736d8d10a926 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.275690 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:26.285422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1208/2415] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:26.288329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1208/2416] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:26 2022] PUT /identity/v3/projects/13fd41934fb346e2a9e578e87856518c/users/56f52f26f52242398fc932c049109af1/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:26.311573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.311573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.311573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.370898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] MFA Rules not processed for user `56f52f26f52242398fc932c049109af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:26.372340 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:26.563387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1209/2417] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:26.623305 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:26.625857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:26.721209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a7240-4e10-477b-967b-e96ed061584c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:26.727866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:26.729616 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.729875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.730351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.748266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.752015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426c736-5621-41f4-9ffe-422a08938a98 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.776530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1209/2418] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:26.795497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:26.941328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d87e76-4bff-49d7-9b05-ccf213bc456d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:26.946978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:26.948540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:26.949104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:26.949487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] PATH_INFO: `/v3/users/56f52f26f52242398fc932c049109af1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:26.950882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f52f26f52242398fc932c049109af1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:26.953024 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:26.975676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:27.058966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f52f26f52242398fc932c049109af1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:27.067867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f52f26f52242398fc932c049109af1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:27.069045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f52f26f52242398fc932c049109af1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:27.069737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f52f26f52242398fc932c049109af1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:27.090962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f52f26f52242398fc932c049109af1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:27.110027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1210/2419] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:26 2022] DELETE /identity/v3/users/56f52f26f52242398fc932c049109af1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:27.114468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:27.115624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:27.116167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:27.143718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:27.144997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:27.219070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1210/2420] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:27.230636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:27.232275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:27.394366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1485-b2dd-40f0-881d-41d55067aa67 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:27.403570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:27.404444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:27.404886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:27.405261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:27.420132 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:27.424080 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c5806-6b38-415e-bbd9-f0f12b32fc7e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:27.440036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1211/2421] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:27.872841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:27.875108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:27.903244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ecf2e-a7d3-42ab-a589-00e42f5de5af None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:27.920104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:27.921183 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:27.921470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:27.921748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] PATH_INFO: `/v3/projects/13fd41934fb346e2a9e578e87856518c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:27.925414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13fd41934fb346e2a9e578e87856518c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:27.927560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:27.991125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56540a9-4df6-4899-9085-417a2582e395 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:27.991443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56540a9-4df6-4899-9085-417a2582e395 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:27.991718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56540a9-4df6-4899-9085-417a2582e395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.014561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a56540a9-4df6-4899-9085-417a2582e395 None None] MFA Rules not processed for user `44db050c06b84991b6a28be2aba39289`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:28.016363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a56540a9-4df6-4899-9085-417a2582e395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:28.053450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13fd41934fb346e2a9e578e87856518c', 'request_id': 'req-d9533e7f-67b4-446e-884b-4a83928d93cd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:28.066243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13fd41934fb346e2a9e578e87856518c', 'request_id': 'req-d9533e7f-67b4-446e-884b-4a83928d93cd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:28.075139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13fd41934fb346e2a9e578e87856518c', 'request_id': 'req-d9533e7f-67b4-446e-884b-4a83928d93cd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:28.095943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] The token cache is being invalidate because project 13fd41934fb346e2a9e578e87856518c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:28.096280 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:28.097345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9533e7f-67b4-446e-884b-4a83928d93cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:28.099447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1211/2422] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:27 2022] DELETE /identity/v3/projects/13fd41934fb346e2a9e578e87856518c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:28.106819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.107148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.107386 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.129634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:28.131631 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:28.132656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1212/2423] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:28.211737 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:28.279084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1212/2424] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:28.281056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.281367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.281653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.301274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:28.302505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:28.322725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.323939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1213/2425] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:28.333937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcbe00-d7df-4aab-b2e0-0051675187c8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.339481 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.340148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.340437 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.340749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.341484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:28.341790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.343761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.362166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1213/2426] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:03:28.363678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.375424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865130f7-4057-4453-bfcf-15ca20ceec2e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.386522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.387300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.387760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.388137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.389077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.391731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.401549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:28.424506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1214/2427] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:28 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:28.428921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.441800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75533b8e-e20e-4393-beac-14f5c1f28870 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.533627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.534217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.534590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.535141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.535848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.538011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.570302 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1214/2428] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:28 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:28.571589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.584578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2385ba-103b-44e3-9d76-e739dc8a786b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.591586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.592264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.592543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.592849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.593582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.595263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.612268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1215/2429] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:28.613606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.625542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ba444-e7ad-476a-8c86-b1e6328f0822 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.632196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.632984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.633350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.633709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.634569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.636368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402d853-ebea-473d-8f66-d64b982bab91 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.654860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1215/2430] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:28.657691 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.668970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe797ae-f769-4a42-a43c-b2c7d8a05ea9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:28.675806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:28.676990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.677485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.677959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] PATH_INFO: `/v3/projects/4704fef3c81a4f1f9db2a18208afd2b7/users/7ecfab0be4984d708fe37352e4d87978/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.697842 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4704fef3c81a4f1f9db2a18208afd2b7, user_id=7ecfab0be4984d708fe37352e4d87978, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:28.700975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:28.734115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:28.735152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:28.735152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:28.736150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1216/2431] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:28 2022] PUT /identity/v3/projects/4704fef3c81a4f1f9db2a18208afd2b7/users/7ecfab0be4984d708fe37352e4d87978/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:28.774500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] MFA Rules not processed for user `7ecfab0be4984d708fe37352e4d87978`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:28.776262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:28.877717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1216/2432] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:28.891286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:28.893614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:29.012587 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51d5e-ea04-431b-861d-482568b533ec admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:29.038975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:29.041440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:29.041742 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:29.045341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:29.066127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:29.068181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97a31d-e670-473f-8af2-df4ee9f7782a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:29.091088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1217/2433] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:31.229750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:31.232046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:31.280255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:31.336344 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cab93-e6e3-475f-accd-ed7887d63733 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:31.341765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:31.342748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:31.343110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:31.343593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] PATH_INFO: `/v3/users/cc535000161d4ba0bca516d56407b3ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:31.344966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc535000161d4ba0bca516d56407b3ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:31.347370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:31.418283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc535000161d4ba0bca516d56407b3ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:31.436846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc535000161d4ba0bca516d56407b3ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:31.439257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc535000161d4ba0bca516d56407b3ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:31.451568 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc535000161d4ba0bca516d56407b3ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:31.495711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc535000161d4ba0bca516d56407b3ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:31.519396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1217/2434] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:31 2022] DELETE /identity/v3/users/cc535000161d4ba0bca516d56407b3ec => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:31.521299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:31.521757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:31.522037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:31.553475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:31.554905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:31.684659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1218/2435] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:31.693424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:31.695343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:31.781130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ec9f-1397-4232-952c-1b83255a7468 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:31.787377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:31.788692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:31.788692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:31.789116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:31.804887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:31.807729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdd00a-3529-4b82-a7f4-bdfebd0987ef service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:31.836733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1218/2436] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:32.478976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:32.480952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:32.504326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46aecc-f0e0-4c8f-9681-4c009856bd67 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:32.520605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:32.521517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:32.522078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:32.522078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] PATH_INFO: `/v3/projects/343bf4b601ab47d0a76b5413a64f7e48` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:32.529042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=343bf4b601ab47d0a76b5413a64f7e48)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:32.532686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:32.644128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343bf4b601ab47d0a76b5413a64f7e48', 'request_id': 'req-0b67da91-2058-42a0-a748-dbb9449d1ccb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:32.646453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:32.657040 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343bf4b601ab47d0a76b5413a64f7e48', 'request_id': 'req-0b67da91-2058-42a0-a748-dbb9449d1ccb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:32.670955 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343bf4b601ab47d0a76b5413a64f7e48', 'request_id': 'req-0b67da91-2058-42a0-a748-dbb9449d1ccb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:32.702908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] The token cache is being invalidate because project 343bf4b601ab47d0a76b5413a64f7e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:32.703145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:32.704856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-0b67da91-2058-42a0-a748-dbb9449d1ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:32.708963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1219/2437] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:32 2022] DELETE /identity/v3/projects/343bf4b601ab47d0a76b5413a64f7e48 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:32.715027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:32.716140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:32.717277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:32.762794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:32.765864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:32.919722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:32.976320 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1219/2438] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:32.978733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:32.979061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:32.979354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.034339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:33.036428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:33.050491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1220/2439] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:33.054203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.066821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfaf79-4bfb-4c96-b5c8-dfdb9a735a8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.073470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.074817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.075782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.076324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.077194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:33.077996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.080121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.101720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1220/2440] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:33.108339 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.128812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375105c-2a40-4d6a-a271-4dc5d1fa580c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.135004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.136573 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.137275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.137944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.138872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.141917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.211859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1221/2441] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:33 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:33.216547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.228557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3352c-cf07-4e23-833a-8984669542eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.307653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.308647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.309082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.309522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.310431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.312864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.340929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1221/2442] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:33 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:33.343590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.366046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3045af-8dbc-47fe-b8fd-6d02ff55b02c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.388087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.390297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.391482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.391774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.392530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.396891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.420191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1222/2443] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:33.424221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.436665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291105b5-83a3-4e6f-ac22-18bc6d0881d6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.441998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.442761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.443052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.443452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.444226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.446152 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307496-d471-4b81-82c7-58cf8e4dd49c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.466697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.467248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1222/2444] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:33.480634 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e72ee-3bb6-4043-9496-22a97f0d0716 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.494815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.495725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.496155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.496440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] PATH_INFO: `/v3/projects/04782223aae3407497e3ce71fff0240d/users/6f05415625eb4c18876ac718f1271439/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.522644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04782223aae3407497e3ce71fff0240d, user_id=6f05415625eb4c18876ac718f1271439, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.525897 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.544906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1223/2445] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:33 2022] PUT /identity/v3/projects/04782223aae3407497e3ce71fff0240d/users/6f05415625eb4c18876ac718f1271439/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:33.554938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f8806-3056-45ee-8caa-fe32bb08471e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.555235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f8806-3056-45ee-8caa-fe32bb08471e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.555519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f8806-3056-45ee-8caa-fe32bb08471e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.598025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2c6f8806-3056-45ee-8caa-fe32bb08471e None None] MFA Rules not processed for user `6f05415625eb4c18876ac718f1271439`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:33.601058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2c6f8806-3056-45ee-8caa-fe32bb08471e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:33.708671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1223/2446] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:33.722474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:33.726277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:33.808940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8349-6443-4944-8f49-ac3479efe288 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:33.815993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:33.817096 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:33.817433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:33.817800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:33.828798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:33.830792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:33.840646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1224/2447] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:36.956186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c31c3-a5cf-47e6-9542-6d1b00c2bac4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:36.956781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c31c3-a5cf-47e6-9542-6d1b00c2bac4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:36.956781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c31c3-a5cf-47e6-9542-6d1b00c2bac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:36.976202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9a8c31c3-a5cf-47e6-9542-6d1b00c2bac4 None None] MFA Rules not processed for user `2c61943bdc5442878c001aae0677adff`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:36.977487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a8c31c3-a5cf-47e6-9542-6d1b00c2bac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:37.014316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:37.038887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1224/2448] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:38.752537 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:38.885515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c6571-d4d4-45bb-be84-97712b8d819d service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:38.893920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:38.895162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:38.895701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:38.896273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] PATH_INFO: `/v3/users/6f05415625eb4c18876ac718f1271439` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:38.900477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f05415625eb4c18876ac718f1271439)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:38.903374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:38.928061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:38.968332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f05415625eb4c18876ac718f1271439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:38.976148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f05415625eb4c18876ac718f1271439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:38.976876 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f05415625eb4c18876ac718f1271439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:38.977485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f05415625eb4c18876ac718f1271439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:38.992685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f05415625eb4c18876ac718f1271439'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:39.015492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1225/2449] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:38 2022] DELETE /identity/v3/users/6f05415625eb4c18876ac718f1271439 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:39.017339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6292b-3ae2-4364-8e56-53dccd8b87ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:39.018358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6292b-3ae2-4364-8e56-53dccd8b87ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:39.018745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6292b-3ae2-4364-8e56-53dccd8b87ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:39.043203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ec6292b-3ae2-4364-8e56-53dccd8b87ad None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:39.044457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ec6292b-3ae2-4364-8e56-53dccd8b87ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:39.132385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1225/2450] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:39.143791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:39.145150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:39.256824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11cdc2-2c04-42f5-8b1d-c52b523cf5fb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:39.269333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:39.272712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:39.274813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:39.275391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:39.298187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:39.300039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:39.315123 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1226/2451] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:39.770547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:39.770838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:39.771661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:39.820287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:39.822876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:39.877004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:39.878568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:39.895636 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da014-00f1-489b-9f3b-e8a8fb8b8577 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:39.901009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:39.902130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:39.902657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:39.903125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] PATH_INFO: `/v3/projects/04782223aae3407497e3ce71fff0240d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:39.905020 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04782223aae3407497e3ce71fff0240d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:39.907368 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:39.926097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1226/2452] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:03:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:39.941819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:39.943775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:39.995518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04782223aae3407497e3ce71fff0240d', 'request_id': 'req-660c48cd-42f0-4a7f-8321-6a64dc95e41a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.005773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04782223aae3407497e3ce71fff0240d', 'request_id': 'req-660c48cd-42f0-4a7f-8321-6a64dc95e41a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.029428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04782223aae3407497e3ce71fff0240d', 'request_id': 'req-660c48cd-42f0-4a7f-8321-6a64dc95e41a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.053162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] The token cache is being invalidate because project 04782223aae3407497e3ce71fff0240d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:40.053462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.054156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-660c48cd-42f0-4a7f-8321-6a64dc95e41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.057039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1227/2453] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:39 2022] DELETE /identity/v3/projects/04782223aae3407497e3ce71fff0240d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:03:40.063236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef39bf-cced-4647-98f4-be2abe758b07 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.064225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef39bf-cced-4647-98f4-be2abe758b07 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.064225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef39bf-cced-4647-98f4-be2abe758b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.120953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17ef39bf-cced-4647-98f4-be2abe758b07 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:40.125741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17ef39bf-cced-4647-98f4-be2abe758b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:40.162907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce4157-77f3-44c6-862f-5e3d9e01f4a4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.173510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:40.264672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:40.282398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1228/2454] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:40.284202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.284392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.284909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.314674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:40.319559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:40.342760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1229/2455] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:40.347732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.374904 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207382bb-1203-4da3-adfb-bad3ac087f43 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.395234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.396113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.396506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.398281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.399505 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:40.400203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.405120 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.406030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.406926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.407375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.407812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.413523 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:40.452869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1230/2456] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:40.457679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.491915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7eed8-1a20-4efc-9109-ab97d5c26235 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.500610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.501471 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.501855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.502234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.507090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.510285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.565600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1231/2457] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:40 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:40.571141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.575676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:40.584953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.587217 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.598504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1227/2458] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:03:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:03:40.599579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.614327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee67539-9df1-488e-a793-f52423bd5828 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.732084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a23f-83b8-4a82-9afa-ff67c5c9dced admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.734592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.735388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.735763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.736050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.736803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.738402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.738845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.739641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.740472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.740936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] PATH_INFO: `/v3/users/7ecfab0be4984d708fe37352e4d87978` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.742345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecfab0be4984d708fe37352e4d87978)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.744716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.778615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1228/2459] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:40 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:40.780063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.829414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dea31d-bfe5-4c35-92df-bb97ac2e1843 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.836898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.838525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.839158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.839579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.840469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.842745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.856447 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1229/2460] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:40.860322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecfab0be4984d708fe37352e4d87978'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.860478 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.873987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746252-2d13-49fb-80f7-9c03aad0b8a8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:40.876472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecfab0be4984d708fe37352e4d87978'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.878414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecfab0be4984d708fe37352e4d87978'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.879429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecfab0be4984d708fe37352e4d87978'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.882918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:40.884788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.886262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.886564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.887340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:40.896348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:40.914085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-597bbc76-fb12-426f-b5a6-9e648444a5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecfab0be4984d708fe37352e4d87978'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:40.919046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:40.924035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1230/2461] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:40.929177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:40.935100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:40.951901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1232/2462] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:40 2022] DELETE /identity/v3/users/7ecfab0be4984d708fe37352e4d87978 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:40.956921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:40.957341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:40.957680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:40.969081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cd51a-0ec3-4044-bc44-18ea161a28e8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:41.003569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:41.004914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:41.039708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:41.117940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:41.119156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:41.119638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:41.120097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] PATH_INFO: `/v3/projects/8515bcfc1ba948edb0a26d0f6279d7fb/users/1e4ddbb4c4304a79bbb31e3b79043899/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:41.138194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8515bcfc1ba948edb0a26d0f6279d7fb, user_id=1e4ddbb4c4304a79bbb31e3b79043899, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:41.142342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f268d-7b12-4dee-89eb-610c74ebd30e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:41.165376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1231/2463] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:40 2022] PUT /identity/v3/projects/8515bcfc1ba948edb0a26d0f6279d7fb/users/1e4ddbb4c4304a79bbb31e3b79043899/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:41.171303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:41.171760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:41.172208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:41.178102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1233/2464] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:03:41.195081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:41.196473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:41.207789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:41.209036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:41.305047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1232/2465] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:41.308515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2128e-478d-4aa3-bb1d-6bfd981abf13 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:41.317477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:41.324343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:41.423947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:41.425370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:41.425881 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:41.426184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:41.451799 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3a6d0-4808-4053-b94a-3e6bedd77f37 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:41.460580 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:41.461600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:41.462725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:41.462725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:41.473159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:41.476054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:41.498787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:41.502525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db0499-cbed-4624-9765-bce6f3289c29 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:41.504405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1234/2466] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:41.534659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1233/2467] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:03:41.943046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:41.945436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:41.977765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562afdf-a064-42e4-aa39-97bdd002e58a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:41.984275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:41.985370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:41.986531 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:41.986822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] PATH_INFO: `/v3/projects/4704fef3c81a4f1f9db2a18208afd2b7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:41.988140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4704fef3c81a4f1f9db2a18208afd2b7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:41.990112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.082755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:42.085091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4704fef3c81a4f1f9db2a18208afd2b7', 'request_id': 'req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:42.098859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4704fef3c81a4f1f9db2a18208afd2b7', 'request_id': 'req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:42.132481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4704fef3c81a4f1f9db2a18208afd2b7', 'request_id': 'req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:42.167757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] The token cache is being invalidate because project 4704fef3c81a4f1f9db2a18208afd2b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:42.168182 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:42.169964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-304fc7a2-63d7-45f6-a7f0-1be74f9674c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:42.172248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1235/2468] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:41 2022] DELETE /identity/v3/projects/4704fef3c81a4f1f9db2a18208afd2b7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:42.176956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.177793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.178317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.200871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:42.202901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:42.316812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:42.346077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1234/2469] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:42.348085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.348671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.349184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.398390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:42.406151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:42.429224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1236/2470] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:42.433764 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.448135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea22863-4a26-4b13-844d-22dfd6fc3a53 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.453676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.454812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.455318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.455801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.457465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:42.458030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.460212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.477292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1235/2471] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:03:42.482692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.505416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41bde4-955d-4812-b3bc-1c5d3ac87e3e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.511853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.512535 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.512869 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.513231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.513954 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.516536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.586560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1237/2472] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:42 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:42.588194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.612122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.614097 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:42.622837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63f812-ca8b-49e3-9eb7-327b9f98d0d3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.813558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.814333 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.814623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.814877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.816743 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.821042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.849497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bbe87-d9af-4667-88cf-3131664d2373 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.862488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.864335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.864800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.865224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] PATH_INFO: `/v3/users/4f18c52a63cc4be682e02a7dbee8a63a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.865988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1236/2473] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:42 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:42.866988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f18c52a63cc4be682e02a7dbee8a63a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.870162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.871002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.889276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066b38-a59b-400c-9173-af3a98512444 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.905599 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.906554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.906999 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.907464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.908367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.912818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.937979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1237/2474] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:42.938912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:42.954633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78b41-d230-46de-825a-60e3b914edd8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:42.960597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:42.964822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.964822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.964822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:42.964822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:42.964822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d73216-7b6c-4fc1-be22-5f19d0af7f0c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:42.991598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1238/2475] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:03:42.991598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fa4f-83a4-4bbd-a9e9-7413191d57e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:42.991598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fa4f-83a4-4bbd-a9e9-7413191d57e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:42.991598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fa4f-83a4-4bbd-a9e9-7413191d57e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.000510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f18c52a63cc4be682e02a7dbee8a63a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:43.021830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f18c52a63cc4be682e02a7dbee8a63a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:43.022855 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f18c52a63cc4be682e02a7dbee8a63a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:43.023617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f18c52a63cc4be682e02a7dbee8a63a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:43.046394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4c58fa4f-83a4-4bbd-a9e9-7413191d57e9 None None] MFA Rules not processed for user `1e4ddbb4c4304a79bbb31e3b79043899`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:43.048103 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4c58fa4f-83a4-4bbd-a9e9-7413191d57e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.050393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f18c52a63cc4be682e02a7dbee8a63a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:43.107274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1238/2476] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:42 2022] DELETE /identity/v3/users/4f18c52a63cc4be682e02a7dbee8a63a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:43.112130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:43.121177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.191204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:43.238306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4baeb8-67a8-4f6e-9bff-d559b0969b5a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:43.251395 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:43.252564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.254555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.255189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] PATH_INFO: `/v3/projects/d3958bfa42cd4ff78cf0748096c1ef78/users/bcd373a5e3d042318d043623c2f53d75/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.258252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1239/2477] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:43.268905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af681-eead-4495-90df-1a3465108144 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.269460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af681-eead-4495-90df-1a3465108144 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.270678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af681-eead-4495-90df-1a3465108144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.327298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3958bfa42cd4ff78cf0748096c1ef78, user_id=bcd373a5e3d042318d043623c2f53d75, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:43.330265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:43.341202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-788af681-eead-4495-90df-1a3465108144 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:43.347230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-788af681-eead-4495-90df-1a3465108144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.362444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1239/2478] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:43 2022] PUT /identity/v3/projects/d3958bfa42cd4ff78cf0748096c1ef78/users/bcd373a5e3d042318d043623c2f53d75/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:43.363381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:43.366559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.372077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1240/2479] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:43.379837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.380278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.380571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.436970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] MFA Rules not processed for user `bcd373a5e3d042318d043623c2f53d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:43.439160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.460600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220042-5cca-4710-8669-ba60f869f3d7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:43.466227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:43.467066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.467618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.468058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.491509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:43.569753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1241/2480] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:43.570617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:43.572849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.587857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:43.589887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:43.604860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1240/2481] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:43.606314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:43.607859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:43.634714 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c78e9a-51d5-4ad7-a950-a2ee06b33521 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:43.645317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:43.647156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.648166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.650409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.672824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:43.679081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:43.709597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1241/2482] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:43.711126 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53a310-d420-4c7d-a112-e21525a07708 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:43.718144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:43.719169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:43.719710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:43.720397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:43.885246 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:43.888173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4b1e7-e0c3-4f14-a72b-c7919caf7bf3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:43.919976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1242/2483] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:44.855849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:44.858049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:44.882088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0496-456a-43d4-9d3f-7fb67c5867bd service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:44.896437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:44.897762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:44.898251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:44.898713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] PATH_INFO: `/v3/projects/a689639fd49c4a4daee68586b7e050c4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:44.901399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a689639fd49c4a4daee68586b7e050c4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:44.903666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.022298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:45.037206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a689639fd49c4a4daee68586b7e050c4', 'request_id': 'req-073e3746-0b48-4cc4-97b7-2d2240e30982'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:45.048527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a689639fd49c4a4daee68586b7e050c4', 'request_id': 'req-073e3746-0b48-4cc4-97b7-2d2240e30982'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:45.075545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a689639fd49c4a4daee68586b7e050c4', 'request_id': 'req-073e3746-0b48-4cc4-97b7-2d2240e30982'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:45.103259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] The token cache is being invalidate because project a689639fd49c4a4daee68586b7e050c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:45.103563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:45.104240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-073e3746-0b48-4cc4-97b7-2d2240e30982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:45.106202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1242/2484] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:44 2022] DELETE /identity/v3/projects/a689639fd49c4a4daee68586b7e050c4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:45.110830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.111345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.111440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.143718 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:45.153630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:45.323501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1243/2485] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:45.323813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.324112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.324379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.351950 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:45.354589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:45.367426 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1243/2486] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:45.371735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.382772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1ff2f-921d-49d4-a771-794665f9f2c1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.389513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.390673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.391185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.391828 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.392884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:03:45.393564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.395629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584bbd2-5142-44eb-a47a-f1717a5539dc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.425789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1244/2487] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:03:45.430465 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.461335 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962e5b4-ec37-4eb7-8577-9aadae77c74b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.481542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.482413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.482934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.483058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.483996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.484413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.484730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.485970 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.489075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.492570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:45.524786 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] MFA Rules not processed for user `1e4ddbb4c4304a79bbb31e3b79043899`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:45.527543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:45.541682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1244/2488] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:45.543495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.639836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4aadf-1165-4649-a6a7-096d4964e72f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.651292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.652324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.652684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.652968 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.654251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.656825 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.678418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1245/2489] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:45.698880 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1245/2490] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:03:45.702885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.723748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b2a82-3301-47a1-ae41-03467a065297 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.731453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.732098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.732387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.732698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.733441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.735048 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.752659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1246/2491] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:45.756780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.799371 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5bc0-d333-490a-8e29-f03d1270df24 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.815809 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.816682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.817075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.817457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.819705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.826540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6c67d-cf64-4ab0-8ab7-be06a4453164 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.872156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1246/2492] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:03:45.872898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:45.895354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938836b-0ca1-4fab-bb3e-939eda891c90 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:45.909161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:45.910074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.910361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.910647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] PATH_INFO: `/v3/projects/8fd30ccd6eea43bc8dbdaf0dc8ab79b8/users/68d4beceb1dd45beb1ee3bbb122d4edf/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:45.951139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd30ccd6eea43bc8dbdaf0dc8ab79b8, user_id=68d4beceb1dd45beb1ee3bbb122d4edf, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:45.953996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:45.996558 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1247/2493] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:03:45 2022] PUT /identity/v3/projects/8fd30ccd6eea43bc8dbdaf0dc8ab79b8/users/68d4beceb1dd45beb1ee3bbb122d4edf/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:45.998217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc05c-dc27-4784-8f1c-eda597966476 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:45.998558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc05c-dc27-4784-8f1c-eda597966476 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:45.998841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc05c-dc27-4784-8f1c-eda597966476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:46.100334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a57fc05c-dc27-4784-8f1c-eda597966476 None None] MFA Rules not processed for user `68d4beceb1dd45beb1ee3bbb122d4edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:46.101680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a57fc05c-dc27-4784-8f1c-eda597966476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:46.327152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1247/2494] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:46.338214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:46.340067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:46.541470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbedc1-e15e-4b66-af22-baa38a59ab8e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:46.545244 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:46.545978 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:46.546321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:46.546614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:46.561848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:46.563777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f20f60-6d54-4efb-8c89-9ec3adca9722 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:46.577846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1248/2495] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:48.205432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:48.206086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:48.206086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:48.231187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] MFA Rules not processed for user `7ef029bafd694b46973cdff0c87123e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:48.246296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:48.263754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:48.360338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1248/2496] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:50.425792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2f0c-f923-45b9-8463-44def26992ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:50.426399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2f0c-f923-45b9-8463-44def26992ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:50.426642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2f0c-f923-45b9-8463-44def26992ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:50.446420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb8b2f0c-f923-45b9-8463-44def26992ad None None] MFA Rules not processed for user `68d4beceb1dd45beb1ee3bbb122d4edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:50.447804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb8b2f0c-f923-45b9-8463-44def26992ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:50.472987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1249/2497] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:53.198613 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:53.200093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:53.267596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e7a23-3bcb-46f7-af1f-784327656e69 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:53.272149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:53.273053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:53.273341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:53.273622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] PATH_INFO: `/v3/users/bcd373a5e3d042318d043623c2f53d75` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:53.275488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd373a5e3d042318d043623c2f53d75)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:53.277481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:53.323914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd373a5e3d042318d043623c2f53d75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:53.330660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd373a5e3d042318d043623c2f53d75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:53.331314 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd373a5e3d042318d043623c2f53d75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:53.331930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd373a5e3d042318d043623c2f53d75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:53.344172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd373a5e3d042318d043623c2f53d75'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:53.359189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1249/2498] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:03:53 2022] DELETE /identity/v3/users/bcd373a5e3d042318d043623c2f53d75 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:53.363773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:53.364390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:53.364390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:53.383257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:53.384454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:53.447591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1250/2499] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:03:53.458590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:53.463155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:53.538402 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:53.562460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d36b-42a7-44bc-b5b0-1c3e44ca400f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:53.567271 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:53.568167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:53.568569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:53.569083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:53.580075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:53.581581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f853e8-e0a5-4489-a83b-60437e631856 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:53.592056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1250/2500] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:03:54.032524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:03:54.037545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:54.089077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2d71e-37c5-443a-93fd-7006dce05b77 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:03:54.110331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:03:54.111208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:54.111495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:54.111759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] PATH_INFO: `/v3/projects/d3958bfa42cd4ff78cf0748096c1ef78` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:54.122767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3958bfa42cd4ff78cf0748096c1ef78)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:03:54.127397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:03:54.146562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:03:54.255395 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3958bfa42cd4ff78cf0748096c1ef78', 'request_id': 'req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:54.266410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3958bfa42cd4ff78cf0748096c1ef78', 'request_id': 'req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:54.279174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3958bfa42cd4ff78cf0748096c1ef78', 'request_id': 'req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:54.295792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] The token cache is being invalidate because project d3958bfa42cd4ff78cf0748096c1ef78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:03:54.296081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:54.296821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7075e5d2-d24a-4f84-a700-851f3d1f3ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:03:54.298605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1251/2501] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:03:54 2022] DELETE /identity/v3/projects/d3958bfa42cd4ff78cf0748096c1ef78 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:03:59.621462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:03:59.622413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:03:59.623331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:03:59.644974 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] MFA Rules not processed for user `44db050c06b84991b6a28be2aba39289`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:03:59.646702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:03:59.732486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1251/2502] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:03:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:04.292846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:05.662961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:05.664055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:05.664853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:05.688428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:05.690299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:05.790881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1252/2503] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:04:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:05.806046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:05.808297 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:05.907145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2b11-d064-415c-9e3b-532c8f2ce95d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:05.914803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:05.915469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:05.915753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:05.916019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:05.933239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:05.934878 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b78c-89b0-4f7c-bb0e-680a9c38084c service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:05.947119 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1252/2504] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:15.770990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=28 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:15.941846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=66 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:17.929709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:17.931760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:18.015286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475de94-22c6-49be-ac10-7f214f1b91c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:18.019409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:18.020144 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:18.020506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:18.020887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:18.022756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:18.099559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:18.101920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:18.112023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1253/2505] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:21.031530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce474-50e1-49e4-ac96-402a2e74e364 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:21.032070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce474-50e1-49e4-ac96-402a2e74e364 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:21.032168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce474-50e1-49e4-ac96-402a2e74e364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:21.114597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-504ce474-50e1-49e4-ac96-402a2e74e364 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:21.115903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-504ce474-50e1-49e4-ac96-402a2e74e364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:21.136996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1253/2506] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:04:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:21.148964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:21.174168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41e23-9ca3-460d-bfc9-3f0fa2788c1f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:21.180508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:21.181429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:21.181732 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:21.182171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:21.196088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:21.197913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:21.207292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1254/2507] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:22.072075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47535-dfc7-4526-a7f8-b4b23243e257 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.073092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47535-dfc7-4526-a7f8-b4b23243e257 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.073718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47535-dfc7-4526-a7f8-b4b23243e257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:22.101542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-72e47535-dfc7-4526-a7f8-b4b23243e257 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:22.104839 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-72e47535-dfc7-4526-a7f8-b4b23243e257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:22.123377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1254/2508] 10.222.0.61 () {58 vars in 967 bytes} [Thu May 19 22:04:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:22.141219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:22.155024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754453b2-95bc-4664-b621-a7b3c5d1f5f6 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:22.161673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:22.162511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.162952 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.163402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:22.173695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:22.175941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:22.191153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1255/2509] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:22.440184 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7594ae-480a-40d8-97f3-7f80d663d44b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.440482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7594ae-480a-40d8-97f3-7f80d663d44b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.440794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7594ae-480a-40d8-97f3-7f80d663d44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:22.459068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0c7594ae-480a-40d8-97f3-7f80d663d44b None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:22.460984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0c7594ae-480a-40d8-97f3-7f80d663d44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:22.469579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1255/2510] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:04:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:22.484052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:22.496383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557de11-26c9-4c92-887e-5fd0595c26a4 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:22.502352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:22.503423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.503864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.504306 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:22.513861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:22.515671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:22.533187 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1256/2511] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:22.908565 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.908903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.909185 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:22.929786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:22.931025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:22.939619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1256/2512] 10.222.0.61 () {58 vars in 967 bytes} [Thu May 19 22:04:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:22.955310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:22.980376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0ae3-a194-46a8-9d2e-9ebb36e1a55a service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:22.991644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:22.992391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:22.992784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:22.993865 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:23.005339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:23.007035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1db45-0801-4c58-b68a-628811ae62f6 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:23.068009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1257/2513] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:25.253960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:25.255477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:25.333571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5301-9993-4cc6-a8fe-6cab53c4fb27 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:25.338201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:25.339077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:25.339362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:25.340259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] PATH_INFO: `/v3/users/68d4beceb1dd45beb1ee3bbb122d4edf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:25.341518 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d4beceb1dd45beb1ee3bbb122d4edf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:25.343509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:25.394667 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d4beceb1dd45beb1ee3bbb122d4edf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.402160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d4beceb1dd45beb1ee3bbb122d4edf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.402902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d4beceb1dd45beb1ee3bbb122d4edf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.403454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d4beceb1dd45beb1ee3bbb122d4edf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.415321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d4beceb1dd45beb1ee3bbb122d4edf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.428560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1257/2514] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:04:25 2022] DELETE /identity/v3/users/68d4beceb1dd45beb1ee3bbb122d4edf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:04:25.433916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:25.434402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:25.434646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:25.453856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:25.455073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:25.500776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:25.518702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1258/2515] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:25.528965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:25.530702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:25.553204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:25.592818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874db3-ed2e-40ad-a45f-3396162ff20d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:25.596491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:25.597170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:25.597455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:25.597730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:25.608488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:25.609978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca28bc3-bf83-4cc4-ac94-db15a15cbb60 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:25.619766 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1258/2516] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:25.798224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:25.799841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:25.820752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2ffb-6efe-4f14-9165-97f83d934f54 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:25.826855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:25.828085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:25.828504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:25.828924 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] PATH_INFO: `/v3/projects/8fd30ccd6eea43bc8dbdaf0dc8ab79b8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:25.831044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd30ccd6eea43bc8dbdaf0dc8ab79b8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:25.833860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:25.889582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd30ccd6eea43bc8dbdaf0dc8ab79b8', 'request_id': 'req-d87900b4-3dc0-455a-9147-d6104843664b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.896552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd30ccd6eea43bc8dbdaf0dc8ab79b8', 'request_id': 'req-d87900b4-3dc0-455a-9147-d6104843664b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.906646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd30ccd6eea43bc8dbdaf0dc8ab79b8', 'request_id': 'req-d87900b4-3dc0-455a-9147-d6104843664b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.921061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] The token cache is being invalidate because project 8fd30ccd6eea43bc8dbdaf0dc8ab79b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:04:25.921512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.922268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d87900b4-3dc0-455a-9147-d6104843664b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:25.924086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1259/2517] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:04:25 2022] DELETE /identity/v3/projects/8fd30ccd6eea43bc8dbdaf0dc8ab79b8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:04:26.698406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:26.698882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:26.698882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:26.700305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1259/2518] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:04:26.707488 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db2499-ff9d-47f1-b4b2-628aa68b4077 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:26.707949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db2499-ff9d-47f1-b4b2-628aa68b4077 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:26.708451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db2499-ff9d-47f1-b4b2-628aa68b4077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:26.787597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-83db2499-ff9d-47f1-b4b2-628aa68b4077 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:26.794349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-83db2499-ff9d-47f1-b4b2-628aa68b4077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:26.899667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1260/2519] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:04:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:26.911447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:26.913373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:26.998432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb43275-cf91-4a59-9f07-d779a6c71d10 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:27.002038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:27.002687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.003025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.003249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.014099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:27.015461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a1f57-bb11-4421-83dc-b318cacafdf3 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:27.029539 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1260/2520] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:27.107871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.108171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.108458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.109884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1261/2521] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:04:27.115163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c368-6870-4ecb-b776-fad9812f22e2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.115522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c368-6870-4ecb-b776-fad9812f22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.115873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c368-6870-4ecb-b776-fad9812f22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.136157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e419c368-6870-4ecb-b776-fad9812f22e2 None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:27.137534 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e419c368-6870-4ecb-b776-fad9812f22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:27.196562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1261/2522] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:04:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:27.208735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:27.211340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:27.225180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e0bc6-2edc-4c20-ba61-864551906800 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:27.229535 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:27.230185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.230469 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.230747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.238213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:27.239627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eeeef-a79a-42b9-8d75-6bb3bcc8c7f2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:27.249449 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1262/2523] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:27.341000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.341313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.341591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.342959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1262/2524] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:04:27.348422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66c97-3a4f-49a2-b118-0f1cd16bd8a2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.348745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66c97-3a4f-49a2-b118-0f1cd16bd8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.349056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66c97-3a4f-49a2-b118-0f1cd16bd8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.368450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-31f66c97-3a4f-49a2-b118-0f1cd16bd8a2 None None] MFA Rules not processed for user `6b6fbc800c444f2aa44f2c33c40149f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:27.369692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-31f66c97-3a4f-49a2-b118-0f1cd16bd8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:27.378859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1263/2525] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:04:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:27.390364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:27.392167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:27.406148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53062f-2b69-4abe-a791-d0919bb274c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:27.409757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:27.410400 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:27.410679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:27.410960 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:27.421085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:27.422516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:27.433433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1263/2526] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:31.263887 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:31.264339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:31.264502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:31.282661 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:31.283794 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:31.289075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:31.343439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1264/2527] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:04:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:31.356218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:31.357655 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:31.413055 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:31.455981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6abe-490d-4c93-8f67-db29b9091f80 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:31.461799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:31.462910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:31.462910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:31.463072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:31.472772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:31.474498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34226b-2e0f-4150-a655-0e3a92908a63 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:31.483966 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1264/2528] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:38.407220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:38.409039 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:38.560489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bbd76-f603-44b7-a92d-8287585b6977 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:38.568324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:38.570660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:38.571238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:38.572482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:38.574759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:38.589218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:38.591000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:38.604036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1265/2529] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:04:38.958415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80fea-7c96-4795-8575-d56e75456893 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:38.959021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80fea-7c96-4795-8575-d56e75456893 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:38.959021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80fea-7c96-4795-8575-d56e75456893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:38.977699 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59f80fea-7c96-4795-8575-d56e75456893 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:38.978929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59f80fea-7c96-4795-8575-d56e75456893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:38.991693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1265/2530] 10.222.0.61 () {58 vars in 1009 bytes} [Thu May 19 22:04:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:39.030234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:39.048295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71669386-6f67-4ff0-bf2b-c5cea7954fb3 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:39.058880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:39.059909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:39.060475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:39.060861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:39.072035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:39.073261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:39.082706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1266/2531] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:48.510161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:48.510959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:48.510959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:48.532597 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:48.534113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:48.542381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1266/2532] 10.222.0.61 () {58 vars in 967 bytes} [Thu May 19 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:48.554581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:48.564994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef08c9e-5f03-4604-91f5-25d28914c7f3 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:48.570619 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:48.571283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:48.571570 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:48.571852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:48.581991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:48.583943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:48.595708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1267/2533] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:04:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:04:50.698455 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:50.699909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:50.765458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed49d1-c7ef-4d04-ab21-5a2564e1f3f2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:50.770019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:50.770883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:50.771168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:50.771446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] PATH_INFO: `/v3/users/44db050c06b84991b6a28be2aba39289` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:50.773204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44db050c06b84991b6a28be2aba39289)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:50.775150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:50.819449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44db050c06b84991b6a28be2aba39289'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:50.826342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44db050c06b84991b6a28be2aba39289'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:50.827002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44db050c06b84991b6a28be2aba39289'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:50.827607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44db050c06b84991b6a28be2aba39289'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:50.840278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7ee05ad6-03ba-4c9a-934f-62648ac632a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44db050c06b84991b6a28be2aba39289'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:50.854111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1267/2534] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:04:50 2022] DELETE /identity/v3/users/44db050c06b84991b6a28be2aba39289 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:04:51.034202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.046463 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3ab04-4d76-4648-97cc-c2dfcfad4b91 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.111143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.112010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.112299 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.112581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] PATH_INFO: `/v3/projects/943870c3b29a4651885aa091a7c39a0e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.113809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] RBAC: Authorizing `identity:delete_project(project_id=943870c3b29a4651885aa091a7c39a0e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.116032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.118379 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:51.170979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943870c3b29a4651885aa091a7c39a0e', 'request_id': 'req-1d6d3095-4ecd-4199-a154-594d84e4366f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:51.178394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943870c3b29a4651885aa091a7c39a0e', 'request_id': 'req-1d6d3095-4ecd-4199-a154-594d84e4366f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:51.188565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943870c3b29a4651885aa091a7c39a0e', 'request_id': 'req-1d6d3095-4ecd-4199-a154-594d84e4366f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:51.203226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] The token cache is being invalidate because project 943870c3b29a4651885aa091a7c39a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:04:51.203591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:51.204434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1d6d3095-4ecd-4199-a154-594d84e4366f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:04:51.206304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1268/2535] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:04:51 2022] DELETE /identity/v3/projects/943870c3b29a4651885aa091a7c39a0e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:04:51.213348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.213675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.213932 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.236646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:51.237944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:51.262387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:04:51.304257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1268/2536] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:51.311839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.312133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.312435 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.337617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:51.338914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:51.349375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1269/2537] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:04:51.354121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.364067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b047b-4cb6-401e-b744-9df22b3d275a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.369033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.369694 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.369976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.370248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.370985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:04:51.371270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.372937 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.388865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1269/2538] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:04:51.392976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.403577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543c9a8-ac89-49b4-804c-32bb6b7b9820 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.408731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.409397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.409810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.410098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.410808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.412741 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.443987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1270/2539] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:04:51 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:04:51.448721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.459083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7272b-c38e-494b-9d3a-ec4e9a3d7dad admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.518793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.519464 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.519752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.520029 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.520751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.522685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.542568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1270/2540] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:04:51 2022] POST /identity/v3/users => generated 280 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:04:51.546814 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.557413 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6283bc-0692-4100-8e0c-0a8037905cab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.561897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.562554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.562839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.563115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.563837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.565465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.577108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1271/2541] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:04:51.581295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.592212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ed40-3486-4a5d-bfee-a4caa2d3ab1b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.596819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.597499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.597798 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.598112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.598921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.600808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9bafc-7207-41c4-ae82-4dce3cf806ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.611405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1271/2542] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:04:51.615577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.626871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f65ab-843f-49bc-8372-dfe382af7fc0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.631689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.633918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.634218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.634849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] PATH_INFO: `/v3/projects/4174bb415f99465f83e9b97969b10963/users/f132e76a73be4b1db62831bb814a7c52/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.656468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4174bb415f99465f83e9b97969b10963, user_id=f132e76a73be4b1db62831bb814a7c52, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.659274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.675020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1272/2543] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:04:51 2022] PUT /identity/v3/projects/4174bb415f99465f83e9b97969b10963/users/f132e76a73be4b1db62831bb814a7c52/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:04:51.686284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da37b-e79c-4369-8201-02f1cbed4724 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.686659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da37b-e79c-4369-8201-02f1cbed4724 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.687015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da37b-e79c-4369-8201-02f1cbed4724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.716099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1a3da37b-e79c-4369-8201-02f1cbed4724 None None] MFA Rules not processed for user `f132e76a73be4b1db62831bb814a7c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:04:51.717373 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1a3da37b-e79c-4369-8201-02f1cbed4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:51.789060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1272/2544] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:04:51 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:04:51.799215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:04:51.801056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:04:51.893972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52541d8b-0cae-4973-ac83-21ad4cf274d7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:04:51.897897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:04:51.898633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:04:51.898862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:04:51.899146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:04:51.907821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:04:51.909291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a21ce-324e-4200-a435-a0fdad8fc457 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:04:51.919062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1273/2545] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:04:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:05:01.783901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:05:01.914724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:05:08.162514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:05:08.163013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:05:08.163110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:05:08.181729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] MFA Rules not processed for user `1e4ddbb4c4304a79bbb31e3b79043899`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:05:08.183111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:05:08.237843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1273/2546] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:05:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:05:09.456530 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:05:09.457098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:05:09.457194 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:05:09.481170 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:05:09.482403 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:05:09.561233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1274/2547] 10.222.0.61 () {58 vars in 967 bytes} [Thu May 19 22:05:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:05:09.576221 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:05:09.577781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:05:09.658802 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e4e0d-b1bf-46e7-94bf-e4122e62c787 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:05:09.664015 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:05:09.664952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:05:09.665194 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:05:09.666165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:05:09.675489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:05:09.677216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:05:09.686177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1274/2548] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:05:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:05:19.552219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:05:19.682674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=57 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:06:34.311542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:06:34.313128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:34.370613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02190f78-9ee2-482a-8104-308e13d363bf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:06:34.374380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:06:34.375025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:06:34.375313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:06:34.375593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:06:34.377384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:34.450415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:06:34.452405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f1b70-65d8-45ff-82f3-2ea1ed3f762e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:06:34.463698 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1275/2549] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:06:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:06:35.641899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:06:35.651103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37a00-88a5-4d08-abc3-dd78a399d453 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:06:35.654867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:06:35.655490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:06:35.655773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:06:35.656046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:06:35.657792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:35.670890 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:06:35.672384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:06:35.683312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1275/2550] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:06:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:06:39.307176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:06:39.307660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:06:39.307754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:06:39.324825 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] MFA Rules not processed for user `7ef029bafd694b46973cdff0c87123e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:06:39.326025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:39.382151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1276/2551] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:06:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:06:45.678462 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:06:49.375620 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:06:55.807366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:06:55.808964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:55.823118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23916-df6e-4ad4-a885-aaf9fe0f6cce service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:06:55.829355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:06:55.829988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:06:55.830269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:06:55.830541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:06:55.832250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:06:55.888763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:06:55.891062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:06:55.900268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1276/2552] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:06:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:07:05.896063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=28 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:07:16.816111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:07:16.817539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:07:16.829646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b673b89-d88e-454a-9220-3c7762737843 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:07:16.833236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:07:16.833881 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:07:16.834171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:07:16.834454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:07:16.836169 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:07:16.847843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:07:16.849624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:07:16.859870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1277/2553] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:07:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:07:25.735330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:07:25.736788 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:07:25.748498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14415d3-48af-4645-b821-3ef8471e7ea7 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:07:25.752975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:07:25.753617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:07:25.753903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:07:25.754186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:07:25.755948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:07:25.766422 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:07:25.767864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:07:25.775848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1277/2554] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:07:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:07:26.854713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:07:35.772301 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:08:21.667476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:08:21.680476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdec73-eb4b-4370-b1aa-da15b7123ff0 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:08:21.685983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:08:21.686609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:08:21.686896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:08:21.687204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:08:21.688991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:08:21.747765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:08:21.749390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:08:21.758282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1278/2555] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:08:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:08:31.754114 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=28 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:08:43.484309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:08:43.494334 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b719d-34f2-437a-be1c-c854d00c8401 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:08:43.497891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:08:43.498524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:08:43.498812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:08:43.499093 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:08:43.500918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:08:43.511577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:08:43.513062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:08:43.522570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1278/2556] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:08:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:08:53.518370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:09:41.984548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:09:41.986000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:09:42.056256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b843-8a73-4f77-b628-0562f5693021 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:09:42.061304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f459db-586c-427b-bf32-46a970080180 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:09:42.061953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f459db-586c-427b-bf32-46a970080180 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:09:42.062241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f459db-586c-427b-bf32-46a970080180 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:09:42.062637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f459db-586c-427b-bf32-46a970080180 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:09:42.064402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-56f459db-586c-427b-bf32-46a970080180 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:09:42.075595 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f459db-586c-427b-bf32-46a970080180 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:09:42.077359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f459db-586c-427b-bf32-46a970080180 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:09:42.088001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1279/2557] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:09:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:09:51.648399 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:09:51.657533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be0a35-b46f-46e5-b688-ab346b3e2804 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:09:51.661128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:09:51.661763 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:09:51.662051 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:09:51.662349 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:09:51.671388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:09:51.672874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:09:51.682052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1279/2558] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:09:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:09:52.083080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=32 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:10:01.677609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:11:56.601154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f459db-586c-427b-bf32-46a970080180 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:11:56.610342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f459db-586c-427b-bf32-46a970080180 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:11:56.613923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:11:56.614505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:11:56.614787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:11:56.615063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:11:56.624432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:11:56.625870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:11:56.653190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1280/2559] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:11:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:12:06.650589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:12:09.125684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:12:09.134746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1335e4-4751-4ad8-8104-cec870d81b7c service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:12:09.138408 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:12:09.139137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:12:09.139430 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:12:09.139712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:12:09.150375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:12:09.151822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:12:09.163048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1280/2560] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:12:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:12:16.548905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:12:16.561225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041c72-1738-4f12-9007-6494159020dd service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:12:16.567432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:12:16.568067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:12:16.568383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:12:16.568695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:12:16.576827 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:12:16.578397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:12:16.605415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1281/2561] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:12:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:12:19.158134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:12:26.602823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:12:43.632331 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:12:43.641614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b66f6-a1bb-43fd-9596-93a165565f4a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:12:43.645204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:12:43.645851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:12:43.646137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:12:43.646414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:12:43.653871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:12:43.655299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:12:43.663615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1281/2562] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:12:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:12:53.659635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:13:22.236276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:13:22.246382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f58f0-73c9-4602-be86-eabddffddff8 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:13:22.249964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:13:22.250594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:13:22.250877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:13:22.251153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:13:22.259451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:13:22.260891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:13:22.288274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1282/2563] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:13:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:13:32.285774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:13:43.560579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:13:43.580720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0f11f-7599-4db0-b1ff-292801032f65 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:13:43.584323 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:13:43.585146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:13:43.585434 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:13:43.585713 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:13:43.609916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:13:43.611300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:13:43.638785 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1282/2564] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:13:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:13:53.636437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=24 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:14:47.029353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:14:47.030845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:14:47.118765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96147611-6c72-46d9-b095-899995b85136 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:14:47.123763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:14:47.124409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:14:47.124737 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:14:47.125057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:14:47.133711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:14:47.135285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:14:47.145448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1283/2565] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:14:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:14:51.873082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:14:51.945682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d085eb-0480-4af8-a8a3-f97f400bfe3b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:14:51.949540 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:14:51.950202 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:14:51.950484 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:14:51.950760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:14:51.952747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:14:52.043997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:14:52.045554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:14:52.066922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1283/2566] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:14:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:14:57.141534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=37 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:15:02.063854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=65 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:16:56.840633 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:16:56.842279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:16:56.867360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a697-fb6f-4e27-8074-ae02f0e201cd service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:16:56.870876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:16:56.871518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:16:56.871806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:16:56.872085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:16:56.873935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:16:56.945405 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:16:56.946823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504aed3-1bc0-42ce-8467-bb0f5ed1569b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:16:56.955749 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1284/2567] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:16:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:06.951659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=38 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:12.015578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:12.017355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.112859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7638e-c557-4fcd-9d34-e68ea460a91b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:12.117481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:12.118347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.118676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.118951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] PATH_INFO: `/v3/users/f132e76a73be4b1db62831bb814a7c52` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.120145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f132e76a73be4b1db62831bb814a7c52)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:12.122106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:12.168161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f132e76a73be4b1db62831bb814a7c52'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.174913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f132e76a73be4b1db62831bb814a7c52'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.175593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f132e76a73be4b1db62831bb814a7c52'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.176203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f132e76a73be4b1db62831bb814a7c52'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.188608 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f132e76a73be4b1db62831bb814a7c52'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.207302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1284/2568] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:17:12 2022] DELETE /identity/v3/users/f132e76a73be4b1db62831bb814a7c52 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:17:12.211747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.212852 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.213394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.242448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:12.243692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.302974 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1285/2569] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:17:12.312969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:12.314405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.370490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:12.395669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da050b-eb62-46ce-9cfb-484ee7b94b92 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:12.399454 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:12.400770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.401057 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.401333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.410778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:12.412218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b9298-24b6-464c-96c9-9ded4e57d464 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:12.423239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1285/2570] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:12.689944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:12.691454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.705164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b5db7-6c5b-4842-b28e-df992f5f443e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:12.709897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:12.710756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.711041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.711321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] PATH_INFO: `/v3/projects/4174bb415f99465f83e9b97969b10963` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.713507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4174bb415f99465f83e9b97969b10963)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:12.715466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:12.765260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4174bb415f99465f83e9b97969b10963', 'request_id': 'req-5139896e-1a1b-478b-8925-4937250a254d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.771643 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4174bb415f99465f83e9b97969b10963', 'request_id': 'req-5139896e-1a1b-478b-8925-4937250a254d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.780717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4174bb415f99465f83e9b97969b10963', 'request_id': 'req-5139896e-1a1b-478b-8925-4937250a254d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.792911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] The token cache is being invalidate because project 4174bb415f99465f83e9b97969b10963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:17:12.793211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.793856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5139896e-1a1b-478b-8925-4937250a254d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:17:12.795396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1286/2571] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:17:12 2022] DELETE /identity/v3/projects/4174bb415f99465f83e9b97969b10963 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:17:12.801478 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.801774 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.802063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.850215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:12.851437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.912702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1286/2572] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:12.920349 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.920658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.920940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.939423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:12.940572 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:12.949354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1287/2573] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:12.953443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:12.962962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e178e3-9289-4289-8c27-f39d118747d1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:12.967556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:12.968207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:12.968492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:12.968800 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:12.971665 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:17:12.972318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:12.974099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:12.988710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1287/2574] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:17:12.991335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.001369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b80d9-f9b3-4d15-9ec4-f9176aa40a7c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.006035 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.006684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.006968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.007242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.007938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.010540 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.048393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1288/2575] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:17:12 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:17:13.052250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.062929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251a983-a47a-4613-97b2-dcbd46cd20f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.119229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.119888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.120179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.120458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.121183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.123110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.142721 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1288/2576] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:17:13 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:17:13.146905 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.157063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4481ab-05f0-4ad9-8c76-1dca7f9ba03b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.161557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.162201 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.162483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.162758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.163475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.165099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.175682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1289/2577] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:17:13.179403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.190068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa085-6a09-4335-9964-9b6d278eac90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.194631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.195269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.195553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.195832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.196647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.198342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b1264-f4c0-4dd7-9f36-f91012828322 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.208232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1289/2578] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:17:13.212385 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.222665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36d2b-c9a6-4cae-a587-e60eb95d2a80 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.227242 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.228763 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.229044 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.229316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] PATH_INFO: `/v3/projects/2b23855fc0db4450a722731ca017e7ea/users/70229d9e99d74494adac052716c5c9d7/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.244400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b23855fc0db4450a722731ca017e7ea, user_id=70229d9e99d74494adac052716c5c9d7, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.247202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.261459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1290/2579] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:17:13 2022] PUT /identity/v3/projects/2b23855fc0db4450a722731ca017e7ea/users/70229d9e99d74494adac052716c5c9d7/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:17:13.266491 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.266784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.267061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.286117 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:13.287350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:13.297553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:13.346978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1290/2580] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:13.357316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:13.359055 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:13.365857 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:13.417699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b409beb-b372-4c06-855e-ee51bcc2e5a6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:13.421324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:13.421975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:13.422261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:13.422543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:13.432420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:13.433878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d19f8-fa99-4f7f-b610-d480391731d8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:13.443491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1291/2581] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:17:16.837875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:16.839319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:16.904521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944670-f21b-436b-abf0-0d343c012be8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:16.908884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:16.909082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:16.909279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:16.909549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:16.911374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:17.002860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.004393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.015845 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1291/2582] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:17.389233 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.389753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.389861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.419166 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] MFA Rules not processed for user `70229d9e99d74494adac052716c5c9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:17.420366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:17.489904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1292/2583] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:17.494042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.504656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d85d9-8f54-4666-9e9c-9241c857886d service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.509248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.509912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.510201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.510479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.511206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.513174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.546236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1292/2584] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:17:17 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:17:17.547510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.558844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8a3a-ff32-40f8-a261-82420001afc5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.623705 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.624392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.624716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.625004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.625709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.627711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.657994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1293/2585] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:17:17 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:17:17.659232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.672436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42ae7f-3d19-4f0e-b6f0-28a06b5235eb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.679567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.680336 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.680643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.680984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.681832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.683503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.695764 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1293/2586] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:17:17.699993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.711550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b88762-233b-4d39-8f97-2c4cc35499c9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.717153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.718153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.718450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.718720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] PATH_INFO: `/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6/users/c8e2dc499d964afe8d119498d3be4938/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.735010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5e46bc7e4341a685efb06ea490f3e6, user_id=c8e2dc499d964afe8d119498d3be4938, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.737956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.752755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1294/2587] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:17:17 2022] PUT /identity/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6/users/c8e2dc499d964afe8d119498d3be4938/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:17:17.756710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.769442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e46fed-4d25-4a1f-9297-b92463f0c3fc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.799874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:17.843091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.843761 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.844045 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.844326 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.845079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.846685 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecc80-f230-4df4-9f91-e3e06271c170 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.856593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1294/2588] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:17:17.861719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:17.867990 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:17.876702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52bb5-e3e1-4d48-a394-63e068af9dcd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:17.881951 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:17.882854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:17.883140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:17.883419 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] PATH_INFO: `/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6/users/c8e2dc499d964afe8d119498d3be4938/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:17.886188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5e46bc7e4341a685efb06ea490f3e6, user_id=c8e2dc499d964afe8d119498d3be4938, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:17.888894 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:17.901330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1295/2589] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:17:17 2022] PUT /identity/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6/users/c8e2dc499d964afe8d119498d3be4938/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:17:22.416033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e2e7-9bd5-4cbe-804b-d199d74cd1fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:22.416720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e2e7-9bd5-4cbe-804b-d199d74cd1fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:22.416720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e2e7-9bd5-4cbe-804b-d199d74cd1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:22.448492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c2b7e2e7-9bd5-4cbe-804b-d199d74cd1fe None None] MFA Rules not processed for user `c8e2dc499d964afe8d119498d3be4938`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:22.449792 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2b7e2e7-9bd5-4cbe-804b-d199d74cd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:22.541765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1295/2590] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:22.558228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:22.560131 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:22.679491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efef0b-e723-4956-9f61-40205b6e7714 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:22.685804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:22.686486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:22.686805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:22.687082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:22.744484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:22.746133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:22.755439 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1296/2591] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:23.820827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:23.820827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:23.822228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:23.848875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] MFA Rules not processed for user `70229d9e99d74494adac052716c5c9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:17:23.850090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:23.857945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1296/2592] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:17:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:17:27.738914 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:27.740342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:27.808077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149db59-4b77-423a-8174-4a7cf0823153 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:27.811826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:27.812472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:27.812784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:27.813255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:27.814869 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:27.828958 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:27.831278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:27.844587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1297/2593] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:29.028879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:29.039432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3571d-a44a-4a33-8320-7e92e405e049 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:29.043525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:29.044165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:29.044450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:29.044753 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:29.046780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:29.060077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:29.061557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:29.073253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1297/2594] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:37.839177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:39.067384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=60 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:17:49.636792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:17:49.645582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895b1f-6e1e-45f8-8581-69935e747ae8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:17:49.649111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:17:49.649758 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:17:49.650954 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:17:49.651277 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:17:49.653073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:17:49.662907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:17:49.664322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:17:49.672417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1298/2595] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:17:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:17:59.668874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:18:10.264108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:18:10.265915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:18:10.283542 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b522ca-8491-4931-8576-2168453247e5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:18:10.290996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:18:10.291700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:18:10.291990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:18:10.292370 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:18:10.294278 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:18:10.307916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:18:10.310586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:18:10.326835 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1298/2596] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:18:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:18:20.321366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:18:22.429454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:18:22.443362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3268a-603b-4b74-af50-f0da6a834219 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:18:22.449065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:18:22.449710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:18:22.450000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:18:22.450283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:18:22.452298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:18:22.523919 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:18:22.525531 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:18:22.535061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1299/2597] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:18:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:18:32.530927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=33 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:18:44.397152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:18:44.408973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1eba2-ebe7-4d1f-a688-061c75dcc066 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:18:44.414415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:18:44.415106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:18:44.415362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:18:44.415644 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:18:44.417595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:18:44.491520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:18:44.493053 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:18:44.502265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1299/2598] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:18:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:18:54.497990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:19:53.949550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:19:53.952569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:19:54.023214 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317d5d-5853-4737-b1e8-ba7f31e0717c service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:19:54.029221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:19:54.029221 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:19:54.029426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:19:54.030100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:19:54.031544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:19:54.043533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:19:54.045607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:19:54.055414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1300/2599] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:19:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:20:04.051159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=32 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:21:56.420784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:21:56.431152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f421e7-7f2c-4d46-bf21-0cd5740c0760 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:21:56.434867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:21:56.435512 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:21:56.435804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:21:56.436083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:21:56.438234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:21:56.508596 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:21:56.510231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:21:56.520137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1300/2600] 10.222.0.61 () {60 vars in 1396 bytes} [Thu May 19 22:21:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:22:06.517476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=28 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:22:17.331398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:22:17.343800 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda77ad-632f-4dcf-9760-86344a36ab9b service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:22:17.349942 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:22:17.350591 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:22:17.351009 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:22:17.351300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:22:17.419765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:22:17.421541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:22:17.441637 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1301/2601] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:22:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:22:25.227189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:22:25.228602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:22:25.287710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b859b4-a62e-4e10-972e-b9761950c068 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:22:25.291284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:22:25.291934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:22:25.292219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:22:25.292511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:22:25.302256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:22:25.303732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:22:25.313686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1301/2602] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:22:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:22:27.438893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=36 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:22:28.171014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:22:28.180593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ced3c-a933-4640-9831-49ef904c2fea service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:22:28.184197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:22:28.184861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:22:28.185153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:22:28.185434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:22:28.196057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:22:28.197521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:22:28.208979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1302/2603] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:22:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:22:28.572162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:22:28.582224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaa23-06da-47bd-9e23-5915ab923ca7 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:22:28.585995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:22:28.586643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:22:28.586930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:22:28.587213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:22:28.598678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:22:28.600444 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:22:28.612521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1302/2604] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:22:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:22:38.204116 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:22:38.606672 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=32 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:22:56.631414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:22:56.640050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c3a2d-e647-4a22-9358-4500045b92b3 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:22:56.643913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:22:56.644551 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:22:56.644868 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:22:56.645153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:22:56.652250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:22:56.653716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:22:56.661858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1303/2605] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:22:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:23:06.658371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:23:23.232939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:23:23.248255 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f52f-1571-40ef-ad9b-4b9cac69b95b service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:23:23.254977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:23:23.255647 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:23:23.255933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:23:23.256233 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:23:23.265188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:23:23.267328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bcec3-bcb6-4ef4-9402-08968a1e0f8d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:23:23.300673 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1303/2606] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:23:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:23:33.298110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=13 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:23:52.813755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:23:52.830807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911fa27-c200-454f-ac67-6d6d2676e390 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:23:52.836896 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:23:52.837638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:23:52.838122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:23:52.838341 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:23:52.850781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:23:52.852382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:23:52.870745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1304/2607] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:23:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:23:54.623453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:23:54.624046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:23:54.624046 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:23:54.664356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] MFA Rules not processed for user `f07074cdff9148b581a33f12749d0277`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:23:54.665251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:23:54.682261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1304/2608] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:23:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:23:54.979700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:23:54.989223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4945ec-4b79-4204-b232-e196fb0bb1e7 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:23:54.992853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:23:54.993487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:23:54.993777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:23:54.994062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:23:54.995848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:23:55.006033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:23:55.007452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:23:55.015409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1305/2609] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:23:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:23:55.362833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:23:55.375048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91ac56-b8a8-41a6-9594-50120f9a6a89 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:23:55.380777 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:23:55.381429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:23:55.381716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:23:55.381993 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:23:55.383856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:23:55.396342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:23:55.397978 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:23:55.408716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1305/2610] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:23:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:03.982002 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:03.997332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cb610-b330-40b8-8624-669da0be0253 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:04.005337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:04.006325 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:04.006845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:04.007288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:04.009460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:04.042472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:04.044204 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0114e5-1d16-4699-a377-ecbbf261d004 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:04.056569 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1306/2611] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:24:05.404219 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=20 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:07.775449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:07.788162 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8f44a-611c-47ac-b325-a838d66731f8 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:07.793697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:07.794361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:07.794631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:07.794909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:07.796720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:07.809913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:07.811523 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:07.822749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1306/2612] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:09.946431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:09.946940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:09.947287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:10.031605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] MFA Rules not processed for user `d04bed939eac45d2907980fea2c644d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:24:10.033736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:10.098440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1307/2613] 10.222.0.61 () {58 vars in 967 bytes} [Thu May 19 22:24:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:24:10.145118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:10.146517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:10.235308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4f139-f264-40b7-851b-03e5887e0a5c service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:10.241066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1c9cf9b02a41f39f46e878747cea3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:10.241707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:10.241989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:10.242265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:10.250568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:10.252292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:10.260453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1307/2614] 10.222.0.61 () {60 vars in 1384 bytes} [Thu May 19 22:24:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 22:24:10.701235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:10.710930 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fd66d-9748-4aaa-afde-62f199c5d72c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:10.714717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:10.715587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:10.716014 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:10.716304 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:10.725600 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:10.727142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:10.755612 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1308/2615] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:10.783242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:10.793274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5d78-8a4e-484a-92d8-4987e68b70b2 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:10.797668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:10.799014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:10.799302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:10.799724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:10.801682 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:10.866627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:10.868105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab40cda-dfd2-4d0d-a18b-08062388b4e6 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:10.878047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1308/2616] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:14.125265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:14.126851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:14.143679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379ccb3-e3c2-4524-b56a-847b18b44272 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:14.148813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:14.149467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:14.149756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:14.150038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:14.151849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:14.164462 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:14.166081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ea8b3-7af7-40e7-952e-b21e6c3eb7c7 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:14.176745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1309/2617] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:17.670321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:17.670824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:17.670959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:17.672256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1309/2618] 10.222.0.61 () {54 vars in 921 bytes} [Thu May 19 22:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 22:24:17.681457 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:17.681904 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:17.682008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:17.687541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:17.721821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] MFA Rules not processed for user `fb31582d6fc14a7f9ec20d107c445390`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:24:17.723208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:17.734310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1310/2619] 10.222.0.61 () {58 vars in 1018 bytes} [Thu May 19 22:24:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:24:17.741514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:17.743581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:17.760168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:17.764495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b562-2442-4c61-8db8-3127327d1645 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:17.770724 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:17.771744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:17.772180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:17.772365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ed0e-54b6-4c0a-96cf-b372d9530554 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:17.772680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] PATH_INFO: `/v3/users/2c61943bdc5442878c001aae0677adff` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:17.774181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c61943bdc5442878c001aae0677adff)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:17.776442 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:17.777640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:17.778279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:17.778559 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:17.778835 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:17.787388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:17.788992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:17.798921 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1311/2620] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:24:17.830228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:17.832460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c61943bdc5442878c001aae0677adff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:17.841218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c61943bdc5442878c001aae0677adff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:17.842000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c61943bdc5442878c001aae0677adff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:17.842614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c61943bdc5442878c001aae0677adff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:17.854528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c61943bdc5442878c001aae0677adff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:17.866919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1310/2621] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:24:17 2022] DELETE /identity/v3/users/2c61943bdc5442878c001aae0677adff => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:18.043719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:18.055686 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9031dcd-319f-4194-973e-741a0bad24eb service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:18.121141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:18.121889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:18.122287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:18.122472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] PATH_INFO: `/v3/projects/8baeef954dd9469fb906af4017375dff` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:18.123588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8baeef954dd9469fb906af4017375dff)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:18.125515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:18.170272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8baeef954dd9469fb906af4017375dff', 'request_id': 'req-306e7239-4b9d-42d2-8f7f-5926e94644d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.177374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8baeef954dd9469fb906af4017375dff', 'request_id': 'req-306e7239-4b9d-42d2-8f7f-5926e94644d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.185222 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8baeef954dd9469fb906af4017375dff', 'request_id': 'req-306e7239-4b9d-42d2-8f7f-5926e94644d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.196938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] The token cache is being invalidate because project 8baeef954dd9469fb906af4017375dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:24:18.197248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.197862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.199396 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1312/2622] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:24:18 2022] DELETE /identity/v3/projects/8baeef954dd9469fb906af4017375dff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:18.203361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:18.205470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:18.218712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672110-0031-4779-a09b-08a5bf2f1ad9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:18.274429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:18.275306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:18.275589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:18.275870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] PATH_INFO: `/v3/users/0a3bc2942c944831a81e59731beb244b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:18.289463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3bc2942c944831a81e59731beb244b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:18.291433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:18.341677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3bc2942c944831a81e59731beb244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.354602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3bc2942c944831a81e59731beb244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.355351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3bc2942c944831a81e59731beb244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.355972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3bc2942c944831a81e59731beb244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.368107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3bc2942c944831a81e59731beb244b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.387803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1311/2623] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:24:18 2022] DELETE /identity/v3/users/0a3bc2942c944831a81e59731beb244b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:18.584427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:18.585879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:18.599115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7239-4b9d-42d2-8f7f-5926e94644d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:18.657993 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:18.658776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:18.659062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:18.659372 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] PATH_INFO: `/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:18.661275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:18.670846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45bdcf65b6a4a11a78979d6a23c2d6b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:18.674012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:18.719252 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45bdcf65b6a4a11a78979d6a23c2d6b', 'request_id': 'req-3474b88c-41b9-4f15-8ab7-e83100a8d74d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.726082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45bdcf65b6a4a11a78979d6a23c2d6b', 'request_id': 'req-3474b88c-41b9-4f15-8ab7-e83100a8d74d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.732417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45bdcf65b6a4a11a78979d6a23c2d6b', 'request_id': 'req-3474b88c-41b9-4f15-8ab7-e83100a8d74d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.743065 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] The token cache is being invalidate because project e45bdcf65b6a4a11a78979d6a23c2d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:24:18.743358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.743994 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:18.745583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1313/2624] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:24:18 2022] DELETE /identity/v3/projects/e45bdcf65b6a4a11a78979d6a23c2d6b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:25.651626 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:25.653104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:25.687831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:25.713513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ad978-5274-48a7-9c94-870ab9bd0e81 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:25.763409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:25.764061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:25.764345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:25.764645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:25.766483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:25.828250 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:25.829935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:25.839292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1312/2625] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:24:28.740123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:32.193122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:32.194617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:32.208935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474b88c-41b9-4f15-8ab7-e83100a8d74d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:32.214846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:32.215734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:32.216031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:32.216315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] PATH_INFO: `/v3/users/70229d9e99d74494adac052716c5c9d7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:32.217624 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=70229d9e99d74494adac052716c5c9d7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:32.220538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:32.267639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70229d9e99d74494adac052716c5c9d7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.275415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70229d9e99d74494adac052716c5c9d7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.276037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70229d9e99d74494adac052716c5c9d7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.283862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70229d9e99d74494adac052716c5c9d7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.296366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70229d9e99d74494adac052716c5c9d7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.303025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1314/2626] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:24:32 2022] DELETE /identity/v3/users/70229d9e99d74494adac052716c5c9d7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:32.482249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:32.493178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876bced-62a3-4f3b-8868-30198f3c8e7a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:32.555033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:32.555891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:32.556174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:32.556450 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] PATH_INFO: `/v3/projects/2b23855fc0db4450a722731ca017e7ea` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:32.557635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b23855fc0db4450a722731ca017e7ea)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:32.559571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:32.600304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:32.608591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b23855fc0db4450a722731ca017e7ea', 'request_id': 'req-e369c7f5-1939-44ef-b843-8cf0e455414f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.615398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b23855fc0db4450a722731ca017e7ea', 'request_id': 'req-e369c7f5-1939-44ef-b843-8cf0e455414f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.622715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b23855fc0db4450a722731ca017e7ea', 'request_id': 'req-e369c7f5-1939-44ef-b843-8cf0e455414f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.633410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] The token cache is being invalidate because project 2b23855fc0db4450a722731ca017e7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:24:32.633717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.634333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.635850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1313/2627] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:24:32 2022] DELETE /identity/v3/projects/2b23855fc0db4450a722731ca017e7ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:32.639849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:32.641284 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:32.654236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c46d-d349-4ae1-aead-2fd48b4915ec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:32.709880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:32.710670 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:32.710955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:32.711230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] PATH_INFO: `/v3/users/c8e2dc499d964afe8d119498d3be4938` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:32.712430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e2dc499d964afe8d119498d3be4938)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:32.714369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:32.760601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e2dc499d964afe8d119498d3be4938'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.768099 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e2dc499d964afe8d119498d3be4938'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.768746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e2dc499d964afe8d119498d3be4938'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.775709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e2dc499d964afe8d119498d3be4938'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.784160 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:32.790456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e2dc499d964afe8d119498d3be4938'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:32.799042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1315/2628] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:24:32 2022] DELETE /identity/v3/users/c8e2dc499d964afe8d119498d3be4938 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:33.048805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:33.050274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:33.062621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369c7f5-1939-44ef-b843-8cf0e455414f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:33.123917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:33.124821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:33.125110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:33.125388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] PATH_INFO: `/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:33.126585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5e46bc7e4341a685efb06ea490f3e6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:33.128506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:33.183469 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5e46bc7e4341a685efb06ea490f3e6', 'request_id': 'req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:33.190103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5e46bc7e4341a685efb06ea490f3e6', 'request_id': 'req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:33.197875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5e46bc7e4341a685efb06ea490f3e6', 'request_id': 'req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:33.209377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] The token cache is being invalidate because project 8c5e46bc7e4341a685efb06ea490f3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:24:33.209666 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:33.210306 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:24:33.212163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1314/2629] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:24:33 2022] DELETE /identity/v3/projects/8c5e46bc7e4341a685efb06ea490f3e6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:24:42.797641 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:43.206266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:24:56.955892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:24:56.957352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:57.099075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b40cec-b751-4ec9-a37e-cfa638bc3807 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:24:57.156228 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:24:57.156923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:24:57.158321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:24:57.158613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:24:57.160434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:24:57.171756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:24:57.173986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:24:57.183775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1316/2630] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:24:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:25:07.179606 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:28:00.281956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:28:00.283347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:28:00.365550 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedc2c6-349b-4c3c-9582-1c86dd6d7f9f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:28:00.369153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:28:00.369793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:28:00.370076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:28:00.370348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:28:00.372104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:28:00.381546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:28:00.382979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:28:00.390707 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1315/2631] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:28:10.387244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=37 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:28:11.386382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:28:11.387791 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:28:11.473074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3af358-80c9-480a-82d8-9cb209dfc6d1 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:28:11.477041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:28:11.477720 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:28:11.478013 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:28:11.478295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:28:11.480702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:28:11.551323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:28:11.552793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:28:11.562472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1317/2632] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:28:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:28:11.650681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:28:11.659964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b08149-47e4-43bb-92a0-34cc11bed897 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:28:11.664364 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:28:11.665033 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:28:11.665324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:28:11.665600 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:28:11.667354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:28:11.738292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:28:11.739813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:28:11.750095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1316/2633] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:28:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:28:21.558264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=62 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:28:21.745693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:30:02.605870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:30:02.607398 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:30:02.622564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fad75-2190-4fef-a75b-ad52bd7fcd75 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:30:02.627590 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:30:02.628232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:30:02.628524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:30:02.628840 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:30:02.637893 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:30:02.639464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:30:02.649723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1318/2634] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:30:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:30:11.870817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:30:11.879272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407eba7-d75e-4f49-abf5-23caf49bc64e service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:30:11.883489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:30:11.884128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:30:11.884412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:30:11.884718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:30:11.886477 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:30:11.956748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:30:11.958196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:30:11.966943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1317/2635] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:30:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:30:12.645247 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:30:21.962916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:32:12.716139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:32:12.718678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:32:12.730597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aa122-86e2-4786-a24e-12c9dee053b9 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:32:12.734917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:32:12.735565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:32:12.735855 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:32:12.736138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:32:12.737928 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:32:12.828438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:32:12.829838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:32:12.851148 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1319/2636] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:32:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:32:13.962930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:32:13.973429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0924-7363-48b4-bce4-f258147d0246 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:32:13.977160 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:32:13.977914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:32:13.978212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:32:13.978497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:32:13.980269 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:32:13.993678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:32:13.995121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:32:14.005973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1318/2637] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:32:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:32:22.848443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=46 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:32:24.001150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:33:04.258550 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:33:04.267365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b570af-6b45-4817-a4e7-79d83a7a790a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:33:04.270939 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:33:04.271585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:33:04.271862 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:33:04.272139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:33:04.279454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:33:04.281477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:33:04.289477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1320/2638] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:33:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:33:10.142646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:33:10.144116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:33:10.158144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d16fd-d06e-4f02-9575-38a6c51865de service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:33:10.163560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:33:10.164191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:33:10.164475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:33:10.164784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:33:10.173520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:33:10.175126 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:33:10.184337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1319/2639] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:33:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:33:14.285812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:33:20.180042 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:34:12.680454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:34:12.690642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70542fdf-2906-4f1d-ba37-1c36fa90e0e6 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:34:12.694215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:34:12.694843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:34:12.695181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:34:12.695456 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:34:12.703946 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:34:12.705454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:34:12.733683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1321/2640] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:34:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:34:22.730502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:35:02.942868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:35:02.944252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:35:03.089403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c23ce-5db8-429a-a881-08a7e66cf646 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:35:03.098652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:35:03.098918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:35:03.099164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:35:03.099446 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:35:03.101378 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:35:03.114492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:35:03.116086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:35:03.127094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1320/2641] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:35:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:35:13.121767 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=61 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:12.899614 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:12.915621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4e154-230c-4b89-a2e0-51b42307cf93 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:12.919255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:12.920423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:12.920751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:12.921040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:12.929338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:12.930755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:12.959716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1322/2642] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:22.957032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=16 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:24.828468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:24.830318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:24.928893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3209b-c544-4c56-9869-2e9bbf5ca50b service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:24.933705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:24.934355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:24.934637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:24.934909 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:24.936906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] Could not find project: 2b23855fc0db4450a722731ca017e7ea.: keystone.exception.ProjectNotFound: Could not find project: 2b23855fc0db4450a722731ca017e7ea. May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 19 22:36:24.959744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 22:36:24.961393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._validate_token_resources() May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 22:36:24.962887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 22:36:24.964192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 22:36:24.966456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2b23855fc0db4450a722731ca017e7ea. May 19 22:36:24.966456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application May 19 22:36:24.966456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1321/2643] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:24 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 133 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 19 22:36:25.998274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:26.010000 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d001-d8c6-4e7b-a6c9-a8a90476a5cc service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:26.013778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:26.014492 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:26.014842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:26.015674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:26.017516 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:26.108500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:26.110506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:26.131911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1323/2644] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:26.668432 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:26.681990 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522c9ea-1e0f-4166-9858-39d25a63755b service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:26.688114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:26.688914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:26.689206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:26.689489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:26.691311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:26.706110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:26.707760 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:26.718822 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1322/2645] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:26.913227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:26.926331 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec36d76-f9db-413c-ab02-73b38b888445 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:26.931476 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:26.932268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:26.932673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:26.933207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:26.935330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:26.950938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:26.955907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:26.971108 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1324/2646] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:36:27.344570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:27.352196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:27.380087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13003-2897-494e-b9a5-1affc564079c service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:27.398650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:27.399660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:27.400148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:27.400499 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:27.404744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:27.418728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:27.420360 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:27.433043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1323/2647] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:30.272356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:30.282853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac27754-30c3-4730-b3ce-30d2c752b30c service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:30.288097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:30.288809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:30.289102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:30.289384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:30.291319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:30.304074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:30.305964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:30.317625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1325/2648] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:30.335884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:30.337413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:30.352548 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f444675-2f64-4712-a7e2-48531bb55c6c service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:30.358147 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:30.359898 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:30.360295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:30.360846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] PATH_INFO: `/v3/users/f07074cdff9148b581a33f12749d0277` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:30.362295 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07074cdff9148b581a33f12749d0277)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:30.364497 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:30.417060 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07074cdff9148b581a33f12749d0277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:30.425479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07074cdff9148b581a33f12749d0277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:30.426213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07074cdff9148b581a33f12749d0277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:30.426872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07074cdff9148b581a33f12749d0277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:30.440293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07074cdff9148b581a33f12749d0277'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:30.455325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1324/2649] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:36:30 2022] DELETE /identity/v3/users/f07074cdff9148b581a33f12749d0277 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:30.891212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:30.904094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacc6ad-6777-4f39-8421-7c6b3a5c71dd service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:30.962011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:30.962864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:30.963145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:30.963422 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] PATH_INFO: `/v3/projects/2308cb6b98094f2f8fc03c7f13c2d8b0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:30.974356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2308cb6b98094f2f8fc03c7f13c2d8b0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:30.976253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.017625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:31.021366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.022873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.035514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2308cb6b98094f2f8fc03c7f13c2d8b0', 'request_id': 'req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:31.044460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2308cb6b98094f2f8fc03c7f13c2d8b0', 'request_id': 'req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:31.052932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2308cb6b98094f2f8fc03c7f13c2d8b0', 'request_id': 'req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:31.062618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:31.064684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] The token cache is being invalidate because project 2308cb6b98094f2f8fc03c7f13c2d8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:36:31.064980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:31.065628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-359e7da6-b3bb-4aa6-ab0d-27c23a4384d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:31.067172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1326/2650] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:36:30 2022] DELETE /identity/v3/projects/2308cb6b98094f2f8fc03c7f13c2d8b0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:31.072703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e374d-5dde-4f71-bd55-2e2b2a6b6b98 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.073274 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e374d-5dde-4f71-bd55-2e2b2a6b6b98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.073564 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e374d-5dde-4f71-bd55-2e2b2a6b6b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.088943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5fdfd-ed6b-491f-8fab-799e8778b250 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.143572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.143809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f97e374d-5dde-4f71-bd55-2e2b2a6b6b98 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:31.144398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.144712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.144987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.145112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f97e374d-5dde-4f71-bd55-2e2b2a6b6b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.146859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.206790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1327/2651] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:36:31.212999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.214485 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.215031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.215388 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.215947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.225272 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1325/2652] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:36:31.236077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:31.237324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.245404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1328/2653] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:36:31.249552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.259393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949aa0-9b36-4a80-97a8-7927487f87d1 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.264325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.265016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.265361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.265584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.266292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:36:31.266595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.268196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.280486 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1326/2654] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:36:31.286110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.295263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11162856-5eed-4ac7-8bf9-1f2e2703965d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.300585 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.301268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.301548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.301821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.302524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.304415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.346394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1329/2655] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:36:31 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:36:31.347448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.357612 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fe2bd-6a7a-445f-a195-f442412d53d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.418702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.419379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.419668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.419949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.420669 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.422841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.447391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1327/2656] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:36:31 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:31.448665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.460555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe480-3ac8-4e7c-945d-1c6c42dc9476 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.465776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.466768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.467130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.467479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.468285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.470725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.481067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1330/2657] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:31.485270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.495227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868714-54e5-4166-9621-07127b925f23 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.499846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.500693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.501161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.501441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.502264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.503864 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639e0eb-1c1d-4cf4-9614-1e57f33689cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.513801 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1328/2658] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:31.518451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.528744 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c928e8-cffd-4947-9f01-ecbc0c8003d6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.533837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.534792 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.535082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.535369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] PATH_INFO: `/v3/projects/bd6c53acbe0545559dcd27eef0549a94/users/a4fb1bc9e4bd48f1a506ea55871335ba/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.551647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6c53acbe0545559dcd27eef0549a94, user_id=a4fb1bc9e4bd48f1a506ea55871335ba, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.554483 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.576212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1331/2659] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:31 2022] PUT /identity/v3/projects/bd6c53acbe0545559dcd27eef0549a94/users/a4fb1bc9e4bd48f1a506ea55871335ba/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:31.576440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.576440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.576440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.583391 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:31.596856 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:31.598103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.660298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1329/2660] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:31.669813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:31.671587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:31.693982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:31.737328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89da1f-fbac-45b9-b8a0-c10a35931f05 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:31.741596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:31.741776 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:31.742081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:31.742356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:31.752136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:31.753611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:31.763256 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1332/2661] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:34.938527 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:34.940125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:34.957153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62a12c-7008-4f1c-b2f2-34d124e4bfd4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:34.961879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:34.962824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:34.963114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:34.963554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] PATH_INFO: `/v3/users/7ef029bafd694b46973cdff0c87123e2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:34.964851 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef029bafd694b46973cdff0c87123e2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:34.967918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:35.028153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef029bafd694b46973cdff0c87123e2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.038125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef029bafd694b46973cdff0c87123e2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.039281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef029bafd694b46973cdff0c87123e2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.040085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef029bafd694b46973cdff0c87123e2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.057614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd3494cc-bd3e-42de-baf0-d2104a2c0899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef029bafd694b46973cdff0c87123e2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.077411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1330/2662] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:36:34 2022] DELETE /identity/v3/users/7ef029bafd694b46973cdff0c87123e2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:35.455147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:35.465948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b695-e298-466b-90bd-f8a8f4182b55 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:35.556927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:35.558629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:35.559109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:35.559556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] PATH_INFO: `/v3/projects/996e01242d5049279cd562458f77562d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:35.561102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996e01242d5049279cd562458f77562d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:35.564453 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:35.633981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996e01242d5049279cd562458f77562d', 'request_id': 'req-d2b88104-bff5-486c-8e29-0910579b4dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.645319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996e01242d5049279cd562458f77562d', 'request_id': 'req-d2b88104-bff5-486c-8e29-0910579b4dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.660970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996e01242d5049279cd562458f77562d', 'request_id': 'req-d2b88104-bff5-486c-8e29-0910579b4dd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.673891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] The token cache is being invalidate because project 996e01242d5049279cd562458f77562d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:36:35.674238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.674901 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d2b88104-bff5-486c-8e29-0910579b4dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:35.677293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1333/2663] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:36:35 2022] DELETE /identity/v3/projects/996e01242d5049279cd562458f77562d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:35.683908 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:35.684201 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:35.684482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:35.722342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:35.723706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:35.833316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1331/2664] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:35.834996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:35.835285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:35.835571 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:35.866029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:35.867322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:35.888220 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1334/2665] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:36:35.893155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:35.910229 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b27a7-2d5a-4529-b272-6141515443cc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:35.918661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:35.920987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:35.921640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:35.922257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:35.923418 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:36:35.923956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:35.926157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:35.951731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1332/2666] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:36:35.954324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:35.966543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc372-4fc9-4d65-8664-21f00154ab7f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:35.974016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:35.975711 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:35.976021 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:35.976307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:35.977253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:35.980415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.013421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1335/2667] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:36:35 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:36.014621 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.027081 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:36.033751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafb0-cd5e-4e8b-aced-bb206eb28936 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.109158 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.109737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.110142 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.110579 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.111350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.113420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.142079 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1333/2668] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:36:36 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:36.146706 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.161477 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894ef8-5dd9-446c-aa7c-5a5c9f1a12cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.167917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.168602 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.168935 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.169218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.170020 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.172751 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.184521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1336/2669] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:36.189107 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.201675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ee359-6ec1-433f-9656-d57acbb2cdf6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.208292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.209113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.209653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.209863 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.210606 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.212270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39280bc-cd3f-422c-8ddb-ca982221a74b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.227935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1334/2670] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:36.229186 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.244482 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889ea4c-4dc9-49b9-bcfb-6aa5f215f2b1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.251609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.251968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.252461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.252790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] PATH_INFO: `/v3/projects/594f0a0c8b62477abcac0d1484aa7148/users/176b51b139bb40c8b33bffdede6d0de0/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.263577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:36.271735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594f0a0c8b62477abcac0d1484aa7148, user_id=176b51b139bb40c8b33bffdede6d0de0, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.275508 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.294829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1337/2671] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:36 2022] PUT /identity/v3/projects/594f0a0c8b62477abcac0d1484aa7148/users/176b51b139bb40c8b33bffdede6d0de0/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:36.307638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.307638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.307638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.336956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] MFA Rules not processed for user `176b51b139bb40c8b33bffdede6d0de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:36.338262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:36.412196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1335/2672] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:36.416586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.521142 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5372f-c815-4f77-a00b-be07c1152680 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.528589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.529803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.530361 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.531412 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.532819 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.534960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592cf15-d70b-4c52-b690-24020830556d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.582933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1338/2673] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:36:36 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:36.584136 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.596683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8d177-e2d2-4cc7-a355-f3626cbbb999 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.678195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.681998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.690828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.690828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.690828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.690828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.692319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.692636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.692907 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.729526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] MFA Rules not processed for user `a4fb1bc9e4bd48f1a506ea55871335ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:36.730759 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:36.736440 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1336/2674] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:36:36 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:36:36.742860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.753159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:36.762231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b0016-8b99-4be2-976f-3c7749ae8f00 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.769178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.769912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.770144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.770425 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.771390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.772979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.786023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1337/2675] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:36.790500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.802212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37171c0d-5419-450b-bb29-6abca924cfc8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.806963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.807930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.808319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.808698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] PATH_INFO: `/v3/projects/59a4aaad8fb7422faafafde0473fb8a2/users/da0574bacc034e70befeaba2e459ff73/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.825494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a4aaad8fb7422faafafde0473fb8a2, user_id=da0574bacc034e70befeaba2e459ff73, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.831866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1339/2676] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:36:36.833841 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.849161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.851028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:36.855344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1338/2677] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:36 2022] PUT /identity/v3/projects/59a4aaad8fb7422faafafde0473fb8a2/users/da0574bacc034e70befeaba2e459ff73/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:36.856357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.921143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:36.933021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da3da-22d4-49aa-a24a-a4d17ba37a0e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.940236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.941098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.941388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.941668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.942476 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.943334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6700a-bf1d-4cb2-bb28-39cdd82bf485 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.944177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:36.956782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1339/2678] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:36.960678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:36.974420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a282f3f-b3d5-4796-86a2-6a54eb5537e9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:36.989016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:36.990035 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:36.990392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:36.990745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] PATH_INFO: `/v3/projects/59a4aaad8fb7422faafafde0473fb8a2/users/da0574bacc034e70befeaba2e459ff73/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:36.995361 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a4aaad8fb7422faafafde0473fb8a2, user_id=da0574bacc034e70befeaba2e459ff73, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:36.998453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85119a61-79e7-47da-99b3-b9bab5574b15 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:37.010731 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1340/2679] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:36 2022] PUT /identity/v3/projects/59a4aaad8fb7422faafafde0473fb8a2/users/da0574bacc034e70befeaba2e459ff73/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:37.023088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:37.023530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:37.024652 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:37.041255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:37.042087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:37.042384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:37.042673 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:37.060443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] MFA Rules not processed for user `da0574bacc034e70befeaba2e459ff73`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:37.061728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:37.108943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:37.110546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:37.120520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1340/2680] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:36:37.136044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1341/2681] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:37.148241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:37.149722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:37.229118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899924-14f2-4689-b3aa-2df38236399a service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:37.304510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:37.305225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:37.305539 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:37.305820 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:37.314215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:37.316956 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:37.319346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6fd8-f919-4959-b7ae-9130f5db3266 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:37.331282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1341/2682] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:37.423525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:37.434043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0b910-b00f-4900-a690-b5fe428b5f41 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:37.437975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:37.438911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:37.439197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:37.439472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:37.479532 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:37.512590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:37.514180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:37.525860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1342/2683] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:38.533025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:38.533542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:38.533665 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:38.552521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] MFA Rules not processed for user `a4fb1bc9e4bd48f1a506ea55871335ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:38.553746 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:38.609334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1342/2684] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:38.962242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:38.964769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:39.043560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04621476-5671-448d-8752-4c59995a1532 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:39.049064 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:39.049729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:39.050019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:39.050291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:39.052197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:39.065187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:39.066984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:39.080661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1343/2685] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:41.721338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:41.722767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:41.796016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8459-224d-4f4a-a26d-fe3eff9ccaf8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:41.801081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a73449-87fe-4498-989b-179a3087529a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:41.801738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a73449-87fe-4498-989b-179a3087529a service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:41.802029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a73449-87fe-4498-989b-179a3087529a service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:41.802345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a73449-87fe-4498-989b-179a3087529a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:41.804132 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99a73449-87fe-4498-989b-179a3087529a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:41.816695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a73449-87fe-4498-989b-179a3087529a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:41.818383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a73449-87fe-4498-989b-179a3087529a service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:41.829263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1343/2686] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:47.654568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:47.721648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901dd2c-2e86-45e5-91b2-8783717e2160 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:47.726248 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:47.727137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:47.727567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:47.727846 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] PATH_INFO: `/v3/users/176b51b139bb40c8b33bffdede6d0de0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:47.729090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176b51b139bb40c8b33bffdede6d0de0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:47.731032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:47.784039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176b51b139bb40c8b33bffdede6d0de0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:47.791700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176b51b139bb40c8b33bffdede6d0de0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:47.792394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176b51b139bb40c8b33bffdede6d0de0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:47.793092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176b51b139bb40c8b33bffdede6d0de0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:47.806545 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176b51b139bb40c8b33bffdede6d0de0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:47.810351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:47.823433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1344/2687] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:36:47 2022] DELETE /identity/v3/users/176b51b139bb40c8b33bffdede6d0de0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:47.827867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:47.828446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:47.828611 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:47.846699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:47.847897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:47.910635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1344/2688] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:47.920448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:47.922153 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:47.981671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11afc08-f501-4d1b-b69f-76188d237550 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:47.985488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:47.986135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:47.986421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:47.986702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:47.996038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:47.997511 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:48.007199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1345/2689] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:48.280909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:48.282423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:48.294434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02495801-c873-42aa-8a38-adbeb5378282 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:48.299926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:48.300850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:48.301138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:48.301414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] PATH_INFO: `/v3/projects/594f0a0c8b62477abcac0d1484aa7148` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:48.302615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594f0a0c8b62477abcac0d1484aa7148)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:48.305428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:48.339695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:48.357225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594f0a0c8b62477abcac0d1484aa7148', 'request_id': 'req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.363873 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594f0a0c8b62477abcac0d1484aa7148', 'request_id': 'req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.371026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594f0a0c8b62477abcac0d1484aa7148', 'request_id': 'req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.381519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] The token cache is being invalidate because project 594f0a0c8b62477abcac0d1484aa7148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:36:48.381807 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.382427 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.383986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1345/2690] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:36:48 2022] DELETE /identity/v3/projects/594f0a0c8b62477abcac0d1484aa7148 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:36:48.387850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:48.397643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877630d0-f101-413d-b23b-d841684a3f5e service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:48.454016 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:48.454893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:48.455177 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:48.455457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] PATH_INFO: `/v3/users/da0574bacc034e70befeaba2e459ff73` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:48.456659 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0574bacc034e70befeaba2e459ff73)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:48.458628 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:48.510472 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0574bacc034e70befeaba2e459ff73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.518236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0574bacc034e70befeaba2e459ff73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.518948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0574bacc034e70befeaba2e459ff73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.519577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0574bacc034e70befeaba2e459ff73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.532312 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-547efb01-bf22-476c-8a52-4d487782bda8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0574bacc034e70befeaba2e459ff73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.548576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1346/2691] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:36:48 2022] DELETE /identity/v3/users/da0574bacc034e70befeaba2e459ff73 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:48.845838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:48.847420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:48.862546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9fba3-ae3e-447e-abe1-e6c2d0de3623 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:48.921839 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:48.922701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:48.922992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:48.923267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] PATH_INFO: `/v3/projects/59a4aaad8fb7422faafafde0473fb8a2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:48.924465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a4aaad8fb7422faafafde0473fb8a2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:48.926417 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:48.977986 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a4aaad8fb7422faafafde0473fb8a2', 'request_id': 'req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.984795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a4aaad8fb7422faafafde0473fb8a2', 'request_id': 'req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:48.993722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a4aaad8fb7422faafafde0473fb8a2', 'request_id': 'req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:49.005874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] The token cache is being invalidate because project 59a4aaad8fb7422faafafde0473fb8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:36:49.006168 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:49.006808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-60a45e4e-d5ad-456d-ae2f-fcdb864f64b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:36:49.008369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1346/2692] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:36:48 2022] DELETE /identity/v3/projects/59a4aaad8fb7422faafafde0473fb8a2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:49.014149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.014684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.014854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.037516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:49.038737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:49.051947 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:49.101557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1347/2693] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:49.110314 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.110610 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.110897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.129383 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:49.130501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:49.139353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1347/2694] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:49.143415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.153279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45455ef9-4bdb-4900-81ea-97b03fd5cd45 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.158727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.159386 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.159674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.159984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.160748 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:36:49.161058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.162716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.175435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1348/2695] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:36:49.179650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.190587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a9c9e-ec78-40a1-b940-96ac30784169 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.195408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.196073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.196366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.196677 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.197387 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.199319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.229245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1348/2696] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:49.233313 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.246217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320573e4-6ee2-4664-bd41-518c5d3fccff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.313888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.314593 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.314882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.315172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.315874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.318043 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.351245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1349/2697] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:49.356015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.371282 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef010d5-266b-4a92-95e6-c70f8d90440d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.376334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.377286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.377598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.377878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.378636 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.380318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.391864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1349/2698] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:36:49.395980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.407105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caace3-cac5-4a6c-9013-da222ea7cd95 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.412241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.412957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.413245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.413521 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.414252 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.415873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c6d0-8255-465e-851d-af4e78d839a1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.428131 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1350/2699] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:49.432343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.438884 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:49.443026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfeca8-950e-4bf3-891c-0c390d5332ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.449554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.450700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.451174 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.451458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] PATH_INFO: `/v3/projects/8e35940173d140bba84fea09ac0358ec/users/10b8d503aa674464bf38997d2faa5bd5/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.467448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e35940173d140bba84fea09ac0358ec, user_id=10b8d503aa674464bf38997d2faa5bd5, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.472309 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.486151 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1350/2700] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:49 2022] PUT /identity/v3/projects/8e35940173d140bba84fea09ac0358ec/users/10b8d503aa674464bf38997d2faa5bd5/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:49.497362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.497657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.497940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.525017 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] MFA Rules not processed for user `10b8d503aa674464bf38997d2faa5bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:49.526213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:49.589231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1351/2701] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:49.594681 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.668554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7b669-8167-4ead-81e6-0bad85d85935 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.688821 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.689729 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.690129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.690527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.691464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.693587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.725213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1351/2702] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:36:49.727063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.738632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd7d24-7221-4f07-9e2e-f08ad82e9839 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.747751 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:49.809562 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.810283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.810569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.810850 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.812146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.814398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.837568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1352/2703] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:36:49 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:36:49.838875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.850254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908aa23-dcb6-4ba7-8b59-8d0978ebaa0c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.855149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.855810 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.856094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.856371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.857196 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.859626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.871642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1352/2704] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:49.875877 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.886456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68969ae6-198f-4995-83fc-8a8e68926a21 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.891291 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.892170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.892452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.892758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] PATH_INFO: `/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50/users/cd20d9f5e7594b8488da66e625b78362/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.907929 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cd65b4ce9e49228ec94a6bc3718e50, user_id=cd20d9f5e7594b8488da66e625b78362, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.910615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:49.924460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1353/2705] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:49 2022] PUT /identity/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50/users/cd20d9f5e7594b8488da66e625b78362/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:49.928354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:49.938337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78002ab5-d6fd-4fe1-a987-0a520b5b4632 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:49.994652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:49.995315 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:49.995609 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:49.995876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:49.996597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:49.998213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c05c7-862a-4ecf-b1ed-9f952cb5913f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:50.008179 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1353/2706] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:36:50.012569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:50.023617 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff9f8-3778-451d-980a-655972456fc8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:50.028384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:50.029304 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:50.029610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:50.029889 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] PATH_INFO: `/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50/users/cd20d9f5e7594b8488da66e625b78362/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:50.032857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cd65b4ce9e49228ec94a6bc3718e50, user_id=cd20d9f5e7594b8488da66e625b78362, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:50.036586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:50.049738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1354/2707] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:36:50 2022] PUT /identity/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50/users/cd20d9f5e7594b8488da66e625b78362/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:36:50.059971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:50.060269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:50.060555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:50.086566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] MFA Rules not processed for user `cd20d9f5e7594b8488da66e625b78362`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:36:50.087778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:50.128426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:36:50.157848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1354/2708] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:36:50.168989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:50.170726 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:50.229113 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a6b42-6544-4680-be2d-cc9c848e8264 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:50.232684 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:50.233961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:50.234247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:50.234522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:50.293121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:50.294581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:50.303969 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1355/2709] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:51.386662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:51.388259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:51.465249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959580d-48e8-4b88-8fde-3b0f68ea3c16 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:51.471586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69572e-e67e-4257-9984-097993edafcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:51.472288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69572e-e67e-4257-9984-097993edafcc service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:51.472521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69572e-e67e-4257-9984-097993edafcc service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:51.472838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69572e-e67e-4257-9984-097993edafcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:51.474617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-db69572e-e67e-4257-9984-097993edafcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:51.486424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69572e-e67e-4257-9984-097993edafcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:51.487998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69572e-e67e-4257-9984-097993edafcc service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:51.498359 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1355/2710] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:36:55.940176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:36:55.941590 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:36:55.953028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43732b-a85a-408f-9918-983c9b497112 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:36:55.956813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:36:55.957456 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:36:55.957736 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:36:55.958009 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:36:55.967329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:36:55.968817 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86117d-667b-4568-b356-9b3514a9e445 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:36:55.978197 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1356/2711] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:36:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:01.494007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=45 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:05.973984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:18.640787 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69572e-e67e-4257-9984-097993edafcc service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:18.705329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69572e-e67e-4257-9984-097993edafcc service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:18.710019 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:18.710891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:18.711177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:18.711460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] PATH_INFO: `/v3/users/10b8d503aa674464bf38997d2faa5bd5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:18.712708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b8d503aa674464bf38997d2faa5bd5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:18.714618 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:18.760703 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b8d503aa674464bf38997d2faa5bd5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:18.767920 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b8d503aa674464bf38997d2faa5bd5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:18.768603 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b8d503aa674464bf38997d2faa5bd5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:18.775430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b8d503aa674464bf38997d2faa5bd5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:18.787464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b8d503aa674464bf38997d2faa5bd5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:18.793343 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1356/2712] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:18 2022] DELETE /identity/v3/users/10b8d503aa674464bf38997d2faa5bd5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:18.798173 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:18.798695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:18.798865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:18.815964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:18.817176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:18.881151 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1357/2713] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:18.891485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:18.892926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:18.949860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21113002-9fe1-4de9-a900-0446cd3e9329 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:18.955077 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:18.955830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:18.956155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:18.956454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:18.966364 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:18.967806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:18.977301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1357/2714] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:19.242482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:19.243946 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:19.257949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aa994-9d16-4af9-be73-7fa417c914c5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:19.262860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:19.263834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:19.264125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:19.264405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] PATH_INFO: `/v3/projects/8e35940173d140bba84fea09ac0358ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:19.265919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e35940173d140bba84fea09ac0358ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:19.267892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:19.315435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e35940173d140bba84fea09ac0358ec', 'request_id': 'req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.321528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e35940173d140bba84fea09ac0358ec', 'request_id': 'req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.328705 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e35940173d140bba84fea09ac0358ec', 'request_id': 'req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.339457 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] The token cache is being invalidate because project 8e35940173d140bba84fea09ac0358ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:19.339741 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.340362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.342156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1358/2715] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:19 2022] DELETE /identity/v3/projects/8e35940173d140bba84fea09ac0358ec => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:19.346875 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:19.356911 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a59f-6e9c-48e1-b7b4-b8978dbfdba2 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:19.376133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:19.416146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:19.417040 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:19.417347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:19.417617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] PATH_INFO: `/v3/users/cd20d9f5e7594b8488da66e625b78362` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:19.418785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd20d9f5e7594b8488da66e625b78362)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:19.420713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:19.467332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd20d9f5e7594b8488da66e625b78362'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.474374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd20d9f5e7594b8488da66e625b78362'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.474995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd20d9f5e7594b8488da66e625b78362'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.481877 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd20d9f5e7594b8488da66e625b78362'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.493554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ced6173c-8708-4a05-9050-5bac920b4d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd20d9f5e7594b8488da66e625b78362'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.500052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1358/2716] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:19 2022] DELETE /identity/v3/users/cd20d9f5e7594b8488da66e625b78362 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:19.777589 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:19.779025 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:19.792578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b21ed7-c41c-4f38-adb3-0b9ddf4c754f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:19.852757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:19.853570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:19.853853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:19.854128 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] PATH_INFO: `/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:19.855310 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06cd65b4ce9e49228ec94a6bc3718e50)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:19.857273 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:19.889111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:19.906354 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06cd65b4ce9e49228ec94a6bc3718e50', 'request_id': 'req-91f81202-3932-47b1-82e5-3609ac6c98d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.912355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06cd65b4ce9e49228ec94a6bc3718e50', 'request_id': 'req-91f81202-3932-47b1-82e5-3609ac6c98d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.919305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06cd65b4ce9e49228ec94a6bc3718e50', 'request_id': 'req-91f81202-3932-47b1-82e5-3609ac6c98d2'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.930148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] The token cache is being invalidate because project 06cd65b4ce9e49228ec94a6bc3718e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:19.930438 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.931070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-91f81202-3932-47b1-82e5-3609ac6c98d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:19.932602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1359/2717] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:19 2022] DELETE /identity/v3/projects/06cd65b4ce9e49228ec94a6bc3718e50 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:19.938525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:19.939111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:19.939111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:19.957083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:19.958283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:20.015945 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1359/2718] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:20.069941 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.070231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.070514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.089749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:20.091210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:20.100121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1360/2719] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:20.104443 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.114032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20279d8a-3d45-4b73-b699-0da5b5cd97bb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.118630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.119286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.119577 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.119860 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.120596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:37:20.120938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.122537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.134630 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1360/2720] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:37:20.138758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.148506 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680cddc-8e2b-4d95-a39d-dd9610010f80 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.153389 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.154031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.154316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.154594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.155283 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.157300 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.182317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1361/2721] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:20.186556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.201663 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b9d2-d4c2-4a49-b49d-a65e74bc9529 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.229615 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:20.263365 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.264028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.264310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.264584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.266018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.267934 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.286972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1361/2722] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:20.290918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.301048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c301-3382-468d-9f70-757a9ea335c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.305591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.306270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.306551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.306827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.307544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.309178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.320355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1362/2723] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:20.324159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.334597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd365cbd-867e-460e-a32f-74e66bae1db3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.339133 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.339780 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.340063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.340339 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.341471 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.343164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a8864-6148-44ef-af75-c9ecaaf78cae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.353150 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1362/2724] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:20.357317 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.367516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1b3a5-bd43-4b2b-b015-ec97d4b2f68d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.372074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.372977 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.373259 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.373536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] PATH_INFO: `/v3/projects/209b80fe573747059a540db4b5b1ea9b/users/fc4029a725f44f40a1c7a1cdd20b24ac/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.388725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209b80fe573747059a540db4b5b1ea9b, user_id=fc4029a725f44f40a1c7a1cdd20b24ac, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.392342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.405286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1363/2725] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:20 2022] PUT /identity/v3/projects/209b80fe573747059a540db4b5b1ea9b/users/fc4029a725f44f40a1c7a1cdd20b24ac/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:20.415646 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.415943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.416223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.443007 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] MFA Rules not processed for user `fc4029a725f44f40a1c7a1cdd20b24ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:20.444397 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:20.521765 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1363/2726] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:20.526105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.587554 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d11074-a2bc-4ada-8ec9-bfe54c366398 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.592403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.593134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.593419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.593698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.594393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.596318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.621585 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1364/2727] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:37:20.626504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.636992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dd866-ad78-40e6-90b7-b3518d15f2d8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.693777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.694466 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.694789 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.695074 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.695782 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.697756 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.720245 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1364/2728] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:20.723570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.733796 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d952b-5e0d-4940-9f63-8514b5fec0cf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.738525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.739180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.739468 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.739750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.740551 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.742244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.753135 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1365/2729] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:20.757062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.767461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d739845-ddb8-4ea7-8caa-48775b1344d9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.772057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.772964 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.773257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.773566 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] PATH_INFO: `/v3/projects/f3520b141f4342ba9d27f4ddb62b2364/users/3b9e6e9f356449d788e9fde66bd675f4/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.788811 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3520b141f4342ba9d27f4ddb62b2364, user_id=3b9e6e9f356449d788e9fde66bd675f4, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.791464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.805275 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1365/2730] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:20 2022] PUT /identity/v3/projects/f3520b141f4342ba9d27f4ddb62b2364/users/3b9e6e9f356449d788e9fde66bd675f4/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:20.809098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.819127 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c6fb-68de-4eef-9afd-dc206f026369 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.839183 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:20.878091 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.878754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.878951 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.879227 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.879948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.881624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d86ee0-9246-41e2-ba35-e0f6696534e9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.891632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1366/2731] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:20.895842 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:20.901287 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:20.908102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfc2cc-6580-40cb-9f67-7fce720e3c19 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:20.913046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:20.913927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.914213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.914496 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] PATH_INFO: `/v3/projects/f3520b141f4342ba9d27f4ddb62b2364/users/3b9e6e9f356449d788e9fde66bd675f4/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.916461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3520b141f4342ba9d27f4ddb62b2364, user_id=3b9e6e9f356449d788e9fde66bd675f4, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:20.919124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:20.929470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1366/2732] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:20 2022] PUT /identity/v3/projects/f3520b141f4342ba9d27f4ddb62b2364/users/3b9e6e9f356449d788e9fde66bd675f4/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:20.940275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:20.940571 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:20.940887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:20.969623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] MFA Rules not processed for user `3b9e6e9f356449d788e9fde66bd675f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:20.970887 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:21.039305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1367/2733] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:21.051052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:21.052863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:21.109350 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc86d-b229-4ac7-9960-a0ade54974bf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:21.112987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:21.113625 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:21.113910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:21.114188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:21.123082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:21.125278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:21.134788 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1367/2734] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:21.232844 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:21.243362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8fb0a-b3c0-468e-a501-6c52c6f4d05f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:21.246886 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:21.247522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:21.247810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:21.248088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:21.304610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:21.306094 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a263de-75f5-4356-bd8a-ed4e1ea764ce service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:21.315616 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1368/2735] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:23.874949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:23.938012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272a53-f794-460d-9be6-0c384c58b4a1 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:23.942779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:23.943841 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:23.944210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:23.944548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] PATH_INFO: `/v3/users/fc4029a725f44f40a1c7a1cdd20b24ac` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:23.945753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc4029a725f44f40a1c7a1cdd20b24ac)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:23.947696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:23.992573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc4029a725f44f40a1c7a1cdd20b24ac'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.000045 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc4029a725f44f40a1c7a1cdd20b24ac'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.000700 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc4029a725f44f40a1c7a1cdd20b24ac'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.007801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc4029a725f44f40a1c7a1cdd20b24ac'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.020172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc4029a725f44f40a1c7a1cdd20b24ac'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.026171 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1368/2736] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:23 2022] DELETE /identity/v3/users/fc4029a725f44f40a1c7a1cdd20b24ac => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:24.030688 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:24.031296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:24.031533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:24.049643 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:24.050848 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:24.090984 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:24.110451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1369/2737] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:24.119979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:24.121440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:24.135676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:24.180662 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0513f1-2cbd-4d06-b3b5-fac6801e240f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:24.184301 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:24.184975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:24.185268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:24.185549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:24.195104 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:24.196534 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:24.206638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1369/2738] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:24.476786 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:24.478302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:24.491943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9703d3-9563-42a8-b151-7ba8a095a0fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:24.497345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:24.498070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:24.498356 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:24.498633 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] PATH_INFO: `/v3/projects/209b80fe573747059a540db4b5b1ea9b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:24.500362 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209b80fe573747059a540db4b5b1ea9b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:24.502121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:24.551986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209b80fe573747059a540db4b5b1ea9b', 'request_id': 'req-4253c86a-6550-4c82-b14d-cb422a51f240'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.558329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209b80fe573747059a540db4b5b1ea9b', 'request_id': 'req-4253c86a-6550-4c82-b14d-cb422a51f240'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.565791 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209b80fe573747059a540db4b5b1ea9b', 'request_id': 'req-4253c86a-6550-4c82-b14d-cb422a51f240'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.576048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] The token cache is being invalidate because project 209b80fe573747059a540db4b5b1ea9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:24.576338 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.576989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.579161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1370/2739] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:24 2022] DELETE /identity/v3/projects/209b80fe573747059a540db4b5b1ea9b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:24.583057 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:24.592802 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692705c0-4207-4fa5-a692-33c3b88ee04d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:24.649529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:24.650310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:24.650601 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:24.650882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] PATH_INFO: `/v3/users/3b9e6e9f356449d788e9fde66bd675f4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:24.652067 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9e6e9f356449d788e9fde66bd675f4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:24.654024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:24.701164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9e6e9f356449d788e9fde66bd675f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.708929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9e6e9f356449d788e9fde66bd675f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.709567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9e6e9f356449d788e9fde66bd675f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.716747 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9e6e9f356449d788e9fde66bd675f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.728420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95208d0b-011c-4421-88b1-fc555687f0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9e6e9f356449d788e9fde66bd675f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:24.734495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1370/2740] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:24 2022] DELETE /identity/v3/users/3b9e6e9f356449d788e9fde66bd675f4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:25.003678 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.005186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:25.019247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253c86a-6550-4c82-b14d-cb422a51f240 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.081709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.082576 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.082866 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.083146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] PATH_INFO: `/v3/projects/f3520b141f4342ba9d27f4ddb62b2364` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.084337 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3520b141f4342ba9d27f4ddb62b2364)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.086311 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.131274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3520b141f4342ba9d27f4ddb62b2364', 'request_id': 'req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:25.137196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3520b141f4342ba9d27f4ddb62b2364', 'request_id': 'req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:25.144930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3520b141f4342ba9d27f4ddb62b2364', 'request_id': 'req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:25.151566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:25.157778 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] The token cache is being invalidate because project f3520b141f4342ba9d27f4ddb62b2364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:25.158125 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:25.158803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa01e895-a0f3-4879-8e9d-8cf2ad736e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:25.160654 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1371/2741] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:25 2022] DELETE /identity/v3/projects/f3520b141f4342ba9d27f4ddb62b2364 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:25.165850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.166347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.166470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.184404 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:25.185557 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:25.215446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:25.247181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1371/2742] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:25.256082 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.256419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.256732 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.275130 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:25.276329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:25.284112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1372/2743] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:25.288375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.301626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84953e-f226-4c52-961c-b66da6fbc55d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.306268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.306926 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.307216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.307498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.308229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:37:25.308542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.310165 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.322430 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1372/2744] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:37:25.326983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.336808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deba38a-69a7-4a0f-b8a0-e211b29e07d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.341874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.342522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.342810 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.343679 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.344381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.346350 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.371783 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1373/2745] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:25 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:37:25.376560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.386560 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545171a-b1f2-4bcb-81b5-eaf9538bb3a4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.442465 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.450218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.450218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.450218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.450218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.450218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.466918 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1373/2746] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:25 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:25.471041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.481447 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e541e-2000-41a2-b1c9-91fafda32513 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.486474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.487137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.487427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.487709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.488445 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.490103 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.501070 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1374/2747] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:25.505078 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.515627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd2d8-4a72-4e0d-943a-aa11582f32f9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.520272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.520967 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.521253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.521543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.522266 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.523853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c907f24-ac85-49ae-ae87-02c5949eb296 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.534425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1374/2748] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:25.538500 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.548397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77ac2-ee02-4e45-8be6-ec8ffd46de59 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.553146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.553996 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.554285 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.554569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] PATH_INFO: `/v3/projects/d343d28953db40c5abcb198bf106c210/users/46b5fa01576e4f088c65a301dcd538d0/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.569530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d343d28953db40c5abcb198bf106c210, user_id=46b5fa01576e4f088c65a301dcd538d0, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.572168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.585333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1375/2749] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:25 2022] PUT /identity/v3/projects/d343d28953db40c5abcb198bf106c210/users/46b5fa01576e4f088c65a301dcd538d0/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:25.590336 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d6e9-79ad-4f2b-acee-29b433376c15 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.590645 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d6e9-79ad-4f2b-acee-29b433376c15 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.591181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d6e9-79ad-4f2b-acee-29b433376c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.608844 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc09d6e9-79ad-4f2b-acee-29b433376c15 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:25.609985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc09d6e9-79ad-4f2b-acee-29b433376c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:25.668208 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1375/2750] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:25.677754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:25.679437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:25.737063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe73320-30d6-4cb9-af1d-06ff1001a4a7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:25.740635 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:25.741288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:25.741577 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:25.741859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:25.751325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:25.752805 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:25.762134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1376/2751] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:26.379992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.380524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.380846 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.409136 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] MFA Rules not processed for user `46b5fa01576e4f088c65a301dcd538d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:26.410363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:26.440971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:26.481323 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1376/2752] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:26.486287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.495998 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba6802-8196-4885-9c6f-0651c27982ba service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.501461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.502157 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.502536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.503174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.503637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.505660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.539637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1377/2753] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:37:26.540771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.553517 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12ed1d-7966-421a-afbe-5edb3e815a48 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.611239 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.611909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.612195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.612473 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.613207 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.615117 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.635366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1377/2754] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:26 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:26.639411 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.649745 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb6e5-87d6-41e9-a209-70ad0ca77f53 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.654374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.655028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.655318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.655640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.656330 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.658013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.668412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1378/2755] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:26.673259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.683090 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a1dbe-979b-4aec-b400-3b77afe7081b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.688526 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.689598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.689777 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.690056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] PATH_INFO: `/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac/users/a8ceb93f99f64d46b623d4ce7ea85a92/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.704861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd1ba53e515499c9b2f6f3579fe2fac, user_id=a8ceb93f99f64d46b623d4ce7ea85a92, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.707509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.721138 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1378/2756] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:26 2022] PUT /identity/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac/users/a8ceb93f99f64d46b623d4ce7ea85a92/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:26.725031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.735419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f063-20d4-4a0e-8124-88b1192b881c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.793205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.793793 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.794083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.794388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.795168 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.796987 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a40fc3-8fcc-44a3-b798-bd895ef66ad6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.802112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:26.810806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1379/2757] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:26.812062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:26.822512 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b145b-fec5-47b7-8a4f-c41140dddfd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:26.827125 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:26.828520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:26.828830 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:26.829111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] PATH_INFO: `/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac/users/a8ceb93f99f64d46b623d4ce7ea85a92/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:26.831084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd1ba53e515499c9b2f6f3579fe2fac, user_id=a8ceb93f99f64d46b623d4ce7ea85a92, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:26.833736 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:26.844676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1379/2758] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:26 2022] PUT /identity/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac/users/a8ceb93f99f64d46b623d4ce7ea85a92/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:27.493217 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:27.493737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:27.493867 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:27.521205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] MFA Rules not processed for user `a8ceb93f99f64d46b623d4ce7ea85a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:27.522570 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:27.593415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1380/2759] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:27.604565 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:27.606434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:27.666243 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc72550-06cf-47b3-b81c-b8bee92b920b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:27.669804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:27.670470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:27.670761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:27.671648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:27.705316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:27.729906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:27.731329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:27.741063 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1380/2760] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:29.080660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:29.082058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:29.094167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1759-1efe-4517-9524-73339c6b57e4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:29.098416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:29.099054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:29.099343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:29.099624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:29.108546 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:29.110039 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:29.119675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1381/2761] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:37.187442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:37.196987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971377-d368-4e6b-9b7b-82ce04fdf97b service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:37.200772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:37.201489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:37.201779 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:37.202080 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:37.203878 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:37.276420 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:37.277897 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:37.289966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1381/2762] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:39.115525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=40 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:41.390680 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:41.392200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:41.468781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7242f-d3a2-47cd-a7e8-d1afbb1485b2 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:41.474935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:41.475541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:41.475830 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:41.476111 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:41.478058 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:41.490676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:41.492290 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:41.502191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1382/2763] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:41.975318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:42.042592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e22130-5907-4d3b-ab5d-19330e6f614f service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:42.047871 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:42.049340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:42.049648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:42.049923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] PATH_INFO: `/v3/users/46b5fa01576e4f088c65a301dcd538d0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:42.051164 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b5fa01576e4f088c65a301dcd538d0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:42.053146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:42.099648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b5fa01576e4f088c65a301dcd538d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.106795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b5fa01576e4f088c65a301dcd538d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.107411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b5fa01576e4f088c65a301dcd538d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.115068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b5fa01576e4f088c65a301dcd538d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.116180 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:42.129371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b5fa01576e4f088c65a301dcd538d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.135468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1382/2764] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:41 2022] DELETE /identity/v3/users/46b5fa01576e4f088c65a301dcd538d0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:37:42.312329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:42.323327 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5aba4-5149-4ac3-8933-2d3792bfc0e6 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:42.380603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:42.381481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:42.381765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:42.382041 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] PATH_INFO: `/v3/projects/d343d28953db40c5abcb198bf106c210` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:42.383242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d343d28953db40c5abcb198bf106c210)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:42.385164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:42.434008 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd343d28953db40c5abcb198bf106c210', 'request_id': 'req-3da37474-4506-42f9-9de1-e16f2a854b64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.440073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd343d28953db40c5abcb198bf106c210', 'request_id': 'req-3da37474-4506-42f9-9de1-e16f2a854b64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.447321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd343d28953db40c5abcb198bf106c210', 'request_id': 'req-3da37474-4506-42f9-9de1-e16f2a854b64'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.457827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] The token cache is being invalidate because project d343d28953db40c5abcb198bf106c210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:42.458114 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.458765 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.460533 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1383/2765] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:42 2022] DELETE /identity/v3/projects/d343d28953db40c5abcb198bf106c210 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:42.464415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:42.466213 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:42.479864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e23a8-7076-4670-b4cb-7a58cc3dc02f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:42.535675 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:42.536542 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:42.536861 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:42.537146 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] PATH_INFO: `/v3/users/a8ceb93f99f64d46b623d4ce7ea85a92` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:42.538340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ceb93f99f64d46b623d4ce7ea85a92)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:42.540504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:42.587416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ceb93f99f64d46b623d4ce7ea85a92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.595353 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ceb93f99f64d46b623d4ce7ea85a92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.596768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ceb93f99f64d46b623d4ce7ea85a92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.604981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ceb93f99f64d46b623d4ce7ea85a92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.618005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9a19aaf2-c37f-4c01-a7e1-3c27ac366e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ceb93f99f64d46b623d4ce7ea85a92'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.624167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1383/2766] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:42 2022] DELETE /identity/v3/users/a8ceb93f99f64d46b623d4ce7ea85a92 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:42.797605 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:42.799907 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:42.814195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37474-4506-42f9-9de1-e16f2a854b64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:42.832071 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:42.874601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:42.875467 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:42.875752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:42.876027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] PATH_INFO: `/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:42.877481 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd1ba53e515499c9b2f6f3579fe2fac)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:42.879417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:42.925345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd1ba53e515499c9b2f6f3579fe2fac', 'request_id': 'req-f1f57c4a-382a-42a0-abad-4724372ef831'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.931409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd1ba53e515499c9b2f6f3579fe2fac', 'request_id': 'req-f1f57c4a-382a-42a0-abad-4724372ef831'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.938899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd1ba53e515499c9b2f6f3579fe2fac', 'request_id': 'req-f1f57c4a-382a-42a0-abad-4724372ef831'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.949780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] The token cache is being invalidate because project 5cd1ba53e515499c9b2f6f3579fe2fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:42.950069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.950695 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f1f57c4a-382a-42a0-abad-4724372ef831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:42.952210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1384/2767] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:42 2022] DELETE /identity/v3/projects/5cd1ba53e515499c9b2f6f3579fe2fac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:42.957879 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:42.958172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:42.958455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:42.977479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:42.978683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.031458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:43.038584 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1384/2768] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:43.047161 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.047412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.047698 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.066463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:43.067674 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.075675 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1385/2769] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:43.079761 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.090068 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cc5f2-a1f1-4f06-be1b-3dbcff91cc99 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.094854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.095532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.095972 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.096251 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.097023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:37:43.097330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.098906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.110172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1385/2770] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:37:43.114543 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.123507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758013-ccf7-4fd3-b688-8d147297f990 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.128780 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.129419 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.129701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.129975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.130662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.132560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.157991 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1386/2771] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:43 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:43.162269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.172501 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf41c6-fa31-4bca-9179-ed82fcb28e54 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.228297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.228983 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.229286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.229546 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.230232 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.232128 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.251265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1386/2772] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:43 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:43.256828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.266868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45e1b0-0d02-410a-b9eb-7604af61a011 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.272034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.272720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.273003 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.273279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.274000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.275610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.286394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1387/2773] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:43.290286 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.301112 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d885a1-0b99-4d12-9ef3-306433c7d9c3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.305687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.306342 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.306629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.306908 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.307635 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.309253 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963e199-a3c9-46e9-8cee-a46e43d13781 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.319134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1387/2774] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:43.323240 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.333706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5205-4597-4fa5-8300-2750df1721fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.338234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.339098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.339379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.339656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] PATH_INFO: `/v3/projects/430f6dd298fc4c5bb9c101acfd3b9d0a/users/720c01c079ee42f39242740adee41f08/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.354077 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=430f6dd298fc4c5bb9c101acfd3b9d0a, user_id=720c01c079ee42f39242740adee41f08, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.356957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.369918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1388/2775] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:43 2022] PUT /identity/v3/projects/430f6dd298fc4c5bb9c101acfd3b9d0a/users/720c01c079ee42f39242740adee41f08/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:43.380752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.381053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.381334 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.407436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] MFA Rules not processed for user `720c01c079ee42f39242740adee41f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:43.408708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.472317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1388/2776] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:43.483503 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.485230 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.538838 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:43.545181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6998-a8ec-40c4-b63b-3983474163bd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.550525 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.551254 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.551502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.551781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.560697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.562159 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6786c-3896-4bf2-9aea-d2a8f325091b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.572228 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1389/2777] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:43.593498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.651588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:43.658724 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e136a-dcf8-4f5f-acab-e82062e7aec9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.663660 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.664524 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.664838 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.665120 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] PATH_INFO: `/v3/users/720c01c079ee42f39242740adee41f08` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.666317 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=720c01c079ee42f39242740adee41f08)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.668230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.714652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720c01c079ee42f39242740adee41f08'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:43.722812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720c01c079ee42f39242740adee41f08'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:43.723415 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720c01c079ee42f39242740adee41f08'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:43.730367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720c01c079ee42f39242740adee41f08'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:43.742826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720c01c079ee42f39242740adee41f08'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:43.748723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1389/2778] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:37:43 2022] DELETE /identity/v3/users/720c01c079ee42f39242740adee41f08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:43.753095 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba451e0-79ce-4407-b881-c41631897563 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.753388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba451e0-79ce-4407-b881-c41631897563 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.753668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba451e0-79ce-4407-b881-c41631897563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.770561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fba451e0-79ce-4407-b881-c41631897563 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:43.771759 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fba451e0-79ce-4407-b881-c41631897563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.838115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1390/2779] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:43.848863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:43.850293 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:43.908725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a824a42-a122-41b8-9a14-348d0c20b4ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:43.912480 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:43.913199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:43.913468 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:43.913767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:43.922795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:43.924949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610ca59-9f92-4748-b7c4-6a46abdd38aa service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:43.934541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1390/2780] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:44.207171 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba451e0-79ce-4407-b881-c41631897563 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.208686 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fba451e0-79ce-4407-b881-c41631897563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:44.226365 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba451e0-79ce-4407-b881-c41631897563 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.231083 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.231964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.232406 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.232595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] PATH_INFO: `/v3/projects/430f6dd298fc4c5bb9c101acfd3b9d0a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.233935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] RBAC: Authorizing `identity:delete_project(project_id=430f6dd298fc4c5bb9c101acfd3b9d0a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.235735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.282191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430f6dd298fc4c5bb9c101acfd3b9d0a', 'request_id': 'req-2ce21172-0a05-42ba-9187-3ff5a4f1401a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:44.289001 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430f6dd298fc4c5bb9c101acfd3b9d0a', 'request_id': 'req-2ce21172-0a05-42ba-9187-3ff5a4f1401a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:44.296384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430f6dd298fc4c5bb9c101acfd3b9d0a', 'request_id': 'req-2ce21172-0a05-42ba-9187-3ff5a4f1401a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:44.307294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] The token cache is being invalidate because project 430f6dd298fc4c5bb9c101acfd3b9d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:37:44.307586 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:44.308253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2ce21172-0a05-42ba-9187-3ff5a4f1401a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:37:44.309853 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1391/2781] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:37:44 2022] DELETE /identity/v3/projects/430f6dd298fc4c5bb9c101acfd3b9d0a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:44.314970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.315450 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.315647 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.333445 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:44.334629 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:44.401574 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1391/2782] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:44.411245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.412010 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.412298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.431738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:44.432986 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:44.442376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1392/2783] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:37:44.446771 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.457538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d701aca-bd3d-486d-9ae6-5f1dfa13c8c0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.462582 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.463257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.463688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.463981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.464795 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:37:44.465111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.467102 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.480036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1392/2784] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:37:44.484332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.494424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa56c0-7ee6-47be-a7c6-504a39a5af68 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.500121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.500814 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.501106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.502238 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.503209 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.505212 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.535560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1393/2785] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:37:44 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:37:44.539727 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.546648 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:44.552058 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b187-9052-4624-84a3-3bb5a7e4c329 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.615109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.615768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.616053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.616329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.617042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.618968 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.641532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1393/2786] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:37:44 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:37:44.643501 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.654012 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244e14-5966-4af0-a3b3-def35a919a10 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.658610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.659265 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.659549 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.659826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.660594 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.662232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.673379 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1394/2787] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:44.677414 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.687240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f776b-d886-4ea6-90a7-41f899d87f0e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.692357 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.693042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.693329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.693605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.694367 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.695936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c31b28-5b76-4104-8640-5a46b1c1b4e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.706659 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1394/2788] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:37:44.710522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.720712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc4a-d0ed-443c-b8d9-00e66eb99015 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.725191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.726106 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.726390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.726661 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] PATH_INFO: `/v3/projects/42a01a84451b4062aff5c678ab3a034f/users/e55aa95176f4447c9df95a1bb94f5ceb/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.740827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a01a84451b4062aff5c678ab3a034f, user_id=e55aa95176f4447c9df95a1bb94f5ceb, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.743473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.756555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1395/2789] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:37:44 2022] PUT /identity/v3/projects/42a01a84451b4062aff5c678ab3a034f/users/e55aa95176f4447c9df95a1bb94f5ceb/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:37:44.767153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.768223 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.768701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.795854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] MFA Rules not processed for user `e55aa95176f4447c9df95a1bb94f5ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:37:44.797106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:44.864837 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1395/2790] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:37:44.878494 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:44.880332 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:44.891657 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:44.946474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ed19b-1059-4705-830c-6ccf8a7f5272 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:44.950140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de01947b95e746f8a5d6f711d982f97e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:44.950849 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:44.951141 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:44.951416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:44.960427 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:44.961923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:44.971102 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1396/2791] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:46.387216 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:37:46.388753 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:46.446982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824b01b-6019-4a01-a821-fca3f72a2e22 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:37:46.451031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:37:46.451688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:37:46.451976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:37:46.452319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:37:46.454056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:37:46.485472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:46.535804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:37:46.537290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30ed1b-e964-4402-96fb-044da0684c4f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:37:46.549755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1396/2792] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:37:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:37:54.966971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=22 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:37:56.543859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=21 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:38:02.537645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:38:02.603614 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da49b-6f48-4d19-8118-1360b5e6b79c service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:38:02.608498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:38:02.609428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:38:02.609718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:38:02.609999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] PATH_INFO: `/v3/users/e55aa95176f4447c9df95a1bb94f5ceb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:38:02.611206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e55aa95176f4447c9df95a1bb94f5ceb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:38:02.613359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:38:02.661902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e55aa95176f4447c9df95a1bb94f5ceb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:02.668607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e55aa95176f4447c9df95a1bb94f5ceb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:02.669286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e55aa95176f4447c9df95a1bb94f5ceb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:02.669884 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e55aa95176f4447c9df95a1bb94f5ceb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:02.681325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e55aa95176f4447c9df95a1bb94f5ceb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:02.694637 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1397/2793] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:38:02 2022] DELETE /identity/v3/users/e55aa95176f4447c9df95a1bb94f5ceb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:38:02.699172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:38:02.699644 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:38:02.699731 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:38:02.717490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:38:02.718712 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:38:02.776854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1397/2794] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:38:02.786392 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:38:02.787804 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:38:02.866701 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f394-9754-4820-a55e-7a697407871d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:38:02.870382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:38:02.871021 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:38:02.871305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:38:02.871582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:38:02.880965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:38:02.882452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:38:02.887687 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:38:02.897619 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1398/2795] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:38:03.168667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:38:03.170093 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:38:03.184360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c72aae-a352-451a-a77f-84dab2003284 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:38:03.189996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:38:03.190885 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:38:03.191167 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:38:03.191440 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] PATH_INFO: `/v3/projects/42a01a84451b4062aff5c678ab3a034f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:38:03.192657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a01a84451b4062aff5c678ab3a034f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:38:03.194652 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:38:03.241683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a01a84451b4062aff5c678ab3a034f', 'request_id': 'req-32a7b8bb-720f-4b4a-8a90-6f559f675cce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:03.247778 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a01a84451b4062aff5c678ab3a034f', 'request_id': 'req-32a7b8bb-720f-4b4a-8a90-6f559f675cce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:03.255715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a01a84451b4062aff5c678ab3a034f', 'request_id': 'req-32a7b8bb-720f-4b4a-8a90-6f559f675cce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:03.265960 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] The token cache is being invalidate because project 42a01a84451b4062aff5c678ab3a034f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:38:03.266250 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:03.266891 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32a7b8bb-720f-4b4a-8a90-6f559f675cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:38:03.269513 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1398/2796] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:38:03 2022] DELETE /identity/v3/projects/42a01a84451b4062aff5c678ab3a034f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:38:06.586510 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:38:06.596555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc62e3-8876-4318-ad30-a4b0b3908d52 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:38:06.646683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:38:06.647355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:38:06.647639 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:38:06.647916 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:38:06.649708 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:38:06.659843 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:38:06.661322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:38:06.669187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1399/2797] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:38:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:38:13.263056 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:38:16.665631 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=27 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:39:26.869185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:39:26.869734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:39:26.869734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:39:26.888082 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] MFA Rules not processed for user `a4fb1bc9e4bd48f1a506ea55871335ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:39:26.889321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:39:26.944327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1399/2798] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:39:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:39:36.937158 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:40:06.850030 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:06.851428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:06.921414 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842691-4589-421f-ba14-328c150b97b7 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:06.927174 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:06.927827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:06.928116 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:06.928397 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:06.930213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:06.943833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:06.945645 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:06.956812 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1400/2799] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:40:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:40:16.951560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=32 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:40:46.293692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:46.295145 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:46.363499 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac024da2-cd96-4251-8f98-43297befbd3a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:46.368029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:46.368944 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.369227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.369506 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] PATH_INFO: `/v3/users/a4fb1bc9e4bd48f1a506ea55871335ba` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.370716 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fb1bc9e4bd48f1a506ea55871335ba)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:46.372642 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:46.421966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fb1bc9e4bd48f1a506ea55871335ba'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.428882 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fb1bc9e4bd48f1a506ea55871335ba'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.429514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fb1bc9e4bd48f1a506ea55871335ba'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.436683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fb1bc9e4bd48f1a506ea55871335ba'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.450399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d62229fb-3862-4f5c-aacf-9210195c812b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fb1bc9e4bd48f1a506ea55871335ba'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.456346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1400/2800] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:40:46 2022] DELETE /identity/v3/users/a4fb1bc9e4bd48f1a506ea55871335ba => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:40:46.631266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:46.647220 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03cfd5-9dd2-466b-b284-e2b10a38e0b0 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:46.703292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:46.704148 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.704431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.704747 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] PATH_INFO: `/v3/projects/bd6c53acbe0545559dcd27eef0549a94` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.705921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6c53acbe0545559dcd27eef0549a94)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:46.707981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:46.773656 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6c53acbe0545559dcd27eef0549a94', 'request_id': 'req-4acbf678-8bed-4f3e-bce4-08941f8010e5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.781567 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6c53acbe0545559dcd27eef0549a94', 'request_id': 'req-4acbf678-8bed-4f3e-bce4-08941f8010e5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.789355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6c53acbe0545559dcd27eef0549a94', 'request_id': 'req-4acbf678-8bed-4f3e-bce4-08941f8010e5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.800737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] The token cache is being invalidate because project bd6c53acbe0545559dcd27eef0549a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:40:46.801032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.802483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4acbf678-8bed-4f3e-bce4-08941f8010e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:40:46.803994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1401/2801] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:40:46 2022] DELETE /identity/v3/projects/bd6c53acbe0545559dcd27eef0549a94 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:40:46.809324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.809623 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.809906 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.830300 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:40:46.831541 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:46.894345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1401/2802] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:40:46.902591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.902918 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.903274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.922237 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:40:46.923458 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:46.932393 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1402/2803] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:40:46.936714 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:46.946657 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1547151-007e-4ada-b485-e1184d9e9474 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:46.951255 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:46.951909 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.952200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.952479 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.953248 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:40:46.953561 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:46.955147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:46.967076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1402/2804] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:40:46.971320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:46.981347 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052401a-83f2-4f19-bc9a-72a5dbb30e18 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:46.985964 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:46.986611 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:46.986891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:46.987167 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:46.988515 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:46.992152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.022172 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1403/2805] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:40:46 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:40:47.023327 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:47.033455 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cf6b9-79a1-4578-803a-02ddde9b4509 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:47.037634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:40:47.098529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:47.099263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.099545 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.099856 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.100554 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:47.102970 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.122202 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1403/2806] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:40:47 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:40:47.126293 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:47.136948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af8dc-a551-4b9e-94d6-c9b2fb6fb52a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:47.141700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:47.142475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.142762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.143038 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.144367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:47.146253 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.156860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1404/2807] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:40:47.160845 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:47.170804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f7a6-c149-499e-9e3d-cbaca97d7934 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:47.175429 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:47.176081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.176370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.176678 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.177410 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:47.179027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb1764-811a-45d1-bff6-e1bac8544913 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.188929 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1404/2808] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:40:47.193377 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:47.203179 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b585a-55cb-4d91-9e22-bf33e4fc6677 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:47.208885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:47.209754 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.210044 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.210322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] PATH_INFO: `/v3/projects/852112e0baf34abba9bbb42ada24ada9/users/78b72eda275c47bb83340914de410668/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.225388 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852112e0baf34abba9bbb42ada24ada9, user_id=78b72eda275c47bb83340914de410668, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:47.228090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.241032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1405/2809] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:40:47 2022] PUT /identity/v3/projects/852112e0baf34abba9bbb42ada24ada9/users/78b72eda275c47bb83340914de410668/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:40:47.246362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a8af-0766-45cb-9928-065c6506b28f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.246638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a8af-0766-45cb-9928-065c6506b28f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.246923 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a8af-0766-45cb-9928-065c6506b28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.265696 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a211a8af-0766-45cb-9928-065c6506b28f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:40:47.266980 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a211a8af-0766-45cb-9928-065c6506b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:47.326005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1405/2810] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:40:47.335387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:47.337092 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:47.369226 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:40:47.398580 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8db4-348d-441e-bb5d-690ba00ec0d8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:47.402325 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:47.402972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:47.403262 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:47.403544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:47.412556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:47.414032 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:47.423376 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1406/2811] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:40:51.425085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103480-fb1c-4e13-a1a4-436abeb62f69 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:51.426071 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103480-fb1c-4e13-a1a4-436abeb62f69 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:51.426262 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103480-fb1c-4e13-a1a4-436abeb62f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:51.455683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7b103480-fb1c-4e13-a1a4-436abeb62f69 None None] MFA Rules not processed for user `78b72eda275c47bb83340914de410668`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:40:51.456985 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7b103480-fb1c-4e13-a1a4-436abeb62f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:51.540498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1406/2812] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:40:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:40:51.558902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:40:51.562437 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:51.642578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382c3c-c2e2-41d3-b11c-024a682cc1bb service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:40:51.648260 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:40:51.648957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:51.649251 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:51.649536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:51.658345 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:40:51.660563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ccc62-d2ae-4a54-98bf-69deda8bbbbb service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:40:51.670640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1407/2813] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:40:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:40:53.004452 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:40:53.005352 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:40:53.005836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:40:53.025709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] MFA Rules not processed for user `78b72eda275c47bb83340914de410668`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:40:53.026995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:40:53.037227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1407/2814] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:40:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:41:01.666697 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=48 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:41:03.018303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:41:55.109581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:41:55.110076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:41:55.110329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:41:55.128466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] MFA Rules not processed for user `78b72eda275c47bb83340914de410668`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:41:55.129716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:55.136936 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1408/2815] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:41:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:41:55.437073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:41:55.446829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea858c43-bd0f-406e-8367-20755c6395d2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:41:55.450348 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:41:55.450977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:41:55.451257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:41:55.451532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:41:55.453285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:55.463141 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:41:55.464848 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:41:55.473502 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1408/2816] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:41:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:41:55.777934 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:41:55.779355 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:55.852435 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bf45c-f448-44ef-8c8a-9f709a943b38 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:41:55.857507 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:41:55.858154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:41:55.858441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:41:55.858720 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:41:55.860492 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:55.872308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:41:55.873931 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:41:55.883958 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1409/2817] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:41:58.220880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:41:58.223948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:58.246937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163c4c-1978-4f57-91ca-32134d217c4e service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:41:58.253426 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:41:58.254198 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:41:58.254556 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:41:58.255049 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:41:58.256982 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:41:58.271173 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:41:58.273592 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:41:58.286949 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1409/2818] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:41:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:42:04.842073 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:42:04.856156 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba94b6-5b61-4dcb-b106-c2a39e304c2c service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:42:04.861700 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:42:04.862453 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:42:04.862744 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:42:04.863149 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:42:04.866239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:42:04.880553 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:42:04.882329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:42:04.893190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1410/2819] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:42:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:42:08.281154 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:42:14.888568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=41 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:42:44.654226 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:42:44.655664 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:42:44.732514 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942e68-4088-428e-bf13-7cd03e229f49 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:42:44.736094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:42:44.736775 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:42:44.737062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:42:44.737340 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:42:44.740121 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:42:44.771992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:42:44.773451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc10cb-0dc5-47ac-8e5b-0bd074b935c1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:42:44.794230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1410/2820] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:42:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:42:47.369235 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:42:47.386728 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec88a5-2fdb-415b-9dae-37f17a324326 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:42:47.391098 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:42:47.391922 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:42:47.392270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:42:47.392646 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:42:47.394582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:42:47.409870 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:42:47.411821 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:42:47.424104 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1411/2821] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:42:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:42:54.791589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=47 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:42:56.000052 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:42:56.000899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:42:56.000899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:42:56.029613 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] MFA Rules not processed for user `78b72eda275c47bb83340914de410668`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:42:56.031089 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:42:56.041291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1411/2822] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:42:57.419218 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:43:06.022503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:43:09.521733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:43:09.532981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5af3-bfd6-4a39-9c76-b00814586b6f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:43:09.538547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:43:09.539190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:43:09.539854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:43:09.539854 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:43:09.541547 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:43:09.554062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:43:09.555677 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3e75-9ce5-4b09-8fa6-e915401fe7e1 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:43:09.567381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1412/2823] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:43:09.836853 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:43:09.838543 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:43:09.851596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b35b9-db40-40eb-9102-a33954c9a192 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:43:09.855804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:43:09.856658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:43:09.856953 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:43:09.857234 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:43:09.859023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:43:09.870583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:43:09.872028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:43:09.882500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1412/2824] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:43:19.562885 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:43:19.877563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:44:00.397824 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:44:00.398382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:44:00.398382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:44:00.415915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] MFA Rules not processed for user `78b72eda275c47bb83340914de410668`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:44:00.417146 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:44:00.423975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1413/2825] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:44:10.411048 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:12.330354 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:12.331808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.391215 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:12.422161 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15b791-be52-43dd-ba8e-2d0ba69900dd service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:12.422394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e41fa-a854-44f8-8167-2da9e59a7ad1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:12.426997 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:12.427362 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:12.427871 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.428016 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.428925 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.429031 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.429216 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] PATH_INFO: `/v3/users/78b72eda275c47bb83340914de410668` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.429371 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.430404 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b72eda275c47bb83340914de410668)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:12.431177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.432321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:12.443122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:12.444770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:12.455272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1413/2826] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:12.481482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b72eda275c47bb83340914de410668'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.488683 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b72eda275c47bb83340914de410668'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.489367 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b72eda275c47bb83340914de410668'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.489979 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b72eda275c47bb83340914de410668'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.502516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9ef93b9-376c-4eb8-a5fc-a35c3e7a1d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b72eda275c47bb83340914de410668'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.516213 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1414/2827] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:12 2022] DELETE /identity/v3/users/78b72eda275c47bb83340914de410668 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:45:12.686268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:12.698532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384d5c-c4ad-4203-9529-9c6a8f9c1691 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:12.758219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:12.759005 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.759290 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.759567 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] PATH_INFO: `/v3/projects/852112e0baf34abba9bbb42ada24ada9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.760767 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852112e0baf34abba9bbb42ada24ada9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:12.762668 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:12.809996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852112e0baf34abba9bbb42ada24ada9', 'request_id': 'req-528f5633-b26c-4c32-b22e-2be0050193b7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.816105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852112e0baf34abba9bbb42ada24ada9', 'request_id': 'req-528f5633-b26c-4c32-b22e-2be0050193b7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.825382 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852112e0baf34abba9bbb42ada24ada9', 'request_id': 'req-528f5633-b26c-4c32-b22e-2be0050193b7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.837295 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] The token cache is being invalidate because project 852112e0baf34abba9bbb42ada24ada9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:12.837605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.838261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:12.840324 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1414/2828] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:12 2022] DELETE /identity/v3/projects/852112e0baf34abba9bbb42ada24ada9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:12.845023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb41f6-5626-461f-903a-38123dd89d74 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.845344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb41f6-5626-461f-903a-38123dd89d74 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.845630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb41f6-5626-461f-903a-38123dd89d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.864257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9feb41f6-5626-461f-903a-38123dd89d74 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:12.865588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9feb41f6-5626-461f-903a-38123dd89d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.868438 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:12.869892 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.893694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:12.936474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1415/2829] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:12.945441 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.945816 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.946181 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.949358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f5633-b26c-4c32-b22e-2be0050193b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:12.958562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:12.959224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:12.960368 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:12.960702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:12.964509 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.972428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:12.973775 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:12.978850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:12.980411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:12.984405 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1416/2830] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:12.988945 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:12.991713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1415/2831] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:45:12.999428 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8dd98-7976-4458-a137-26fcef218483 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.004382 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.005076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.005366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.005648 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.007067 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:45:13.007717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.009150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.029988 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1417/2832] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 22:45:13.032489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.045025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874362-017b-4b2d-b46e-7ed8fcdbbe11 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.050159 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.051338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.051338 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.051583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.052105 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.054803 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.086310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1416/2833] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:13.090618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.101857 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3499-8aa6-4e7e-9fec-34fcb5074efe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.141287 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:13.163348 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.164169 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.164461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.164790 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.165516 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.167451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.192061 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1418/2834] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:45:13.193322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.203692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d526e9-0ac0-402b-9cb9-cb9124e027cb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.209793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.210521 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.210890 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.211163 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.211971 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.214010 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.224689 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1417/2835] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:45:13.229488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.239899 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ddc5-2c7a-46c4-9db8-0770ed466273 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.244578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.245276 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.245558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.245836 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.246588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.248203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bc0e9-70c9-41e3-91ad-fefef694e395 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.260050 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1419/2836] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:45:13.264149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.274936 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8285b-0824-4170-a80a-44d0a2198e0c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.279552 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.280428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.280813 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.281100 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] PATH_INFO: `/v3/projects/4ef6cb471e6b4f13b23dac9194663e8e/users/1f628a14f943417d8f12f6bb2361218a/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.296369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef6cb471e6b4f13b23dac9194663e8e, user_id=1f628a14f943417d8f12f6bb2361218a, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.299062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.311651 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1418/2837] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:13 2022] PUT /identity/v3/projects/4ef6cb471e6b4f13b23dac9194663e8e/users/1f628a14f943417d8f12f6bb2361218a/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:13.321660 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.321952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.322249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.348961 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] MFA Rules not processed for user `1f628a14f943417d8f12f6bb2361218a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:13.350837 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:13.414375 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1420/2838] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:13.418876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.480822 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b1f3-50d5-4d90-ad0c-fcdda5b94eab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.485538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.486205 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.486490 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.486768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.487494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.489428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.517127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1419/2839] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:13.521118 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.532268 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baa02a-b776-4aa9-b842-913000fe24a2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.595410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.596088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.596369 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.596676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.597363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.599281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.619124 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1421/2840] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 22:45:13.623069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.633772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1af04-0cc1-4c2e-8b3c-ab8303059c25 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.638345 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.638998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.639281 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.639555 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.640279 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.641889 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.653378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1420/2841] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:13.657343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.667316 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d4724-ed57-46a5-82c0-f65f414f6113 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.671962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.672860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.673150 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.673433 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] PATH_INFO: `/v3/projects/140ea59124494640811747bf9d6c1518/users/e6f7a2f7c33049f28f2a65d227588f29/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.688286 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] RBAC: Authorizing `identity:create_grant(project_id=140ea59124494640811747bf9d6c1518, user_id=e6f7a2f7c33049f28f2a65d227588f29, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.691013 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.703989 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1422/2842] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:13 2022] PUT /identity/v3/projects/140ea59124494640811747bf9d6c1518/users/e6f7a2f7c33049f28f2a65d227588f29/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:13.707981 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.719249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d418da-7367-4e45-8f5f-fb50c230c229 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.743241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:13.778774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.779366 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.779658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.779943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.780695 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.782291 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf97cd-0f4e-49ad-8a6a-56fbcea4a56e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.792118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1421/2843] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:13.796452 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.806602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c012-8f3d-42d6-972a-6ff5ae26014e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:13.811163 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:13.812027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.812309 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.812584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] PATH_INFO: `/v3/projects/140ea59124494640811747bf9d6c1518/users/e6f7a2f7c33049f28f2a65d227588f29/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.814584 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140ea59124494640811747bf9d6c1518, user_id=e6f7a2f7c33049f28f2a65d227588f29, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:13.817258 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:13.823882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:13.829121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1423/2844] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:13 2022] PUT /identity/v3/projects/140ea59124494640811747bf9d6c1518/users/e6f7a2f7c33049f28f2a65d227588f29/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:13.839910 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:13.840210 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:13.840504 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:13.866804 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] MFA Rules not processed for user `e6f7a2f7c33049f28f2a65d227588f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:13.868018 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:13.935697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1422/2845] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:13.947207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:13.948980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:14.011917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e809d-3b01-4a42-8935-a851ea0fd9a2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:14.015588 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:14.016239 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:14.016524 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:14.016832 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:14.029264 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:14.031019 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:14.042099 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1424/2846] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:14.088859 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:14.090259 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:14.102461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8768cdc-02c5-4947-9e6a-b144c9c58985 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:14.106073 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:14.106739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:14.107024 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:14.107303 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:14.165589 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:14.167037 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eb6ac-821e-4b93-beba-6ba543d7a9de service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:14.176109 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1423/2847] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:17.420302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:17.482895 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c373a-d5e1-4257-8418-181a250afbe3 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:17.487342 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:17.488204 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:17.488488 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:17.488795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] PATH_INFO: `/v3/users/1f628a14f943417d8f12f6bb2361218a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:17.489962 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f628a14f943417d8f12f6bb2361218a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:17.492640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:17.536915 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f628a14f943417d8f12f6bb2361218a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:17.544417 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f628a14f943417d8f12f6bb2361218a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:17.545122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f628a14f943417d8f12f6bb2361218a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:17.545735 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f628a14f943417d8f12f6bb2361218a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:17.558188 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f628a14f943417d8f12f6bb2361218a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:17.570953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1425/2848] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:17 2022] DELETE /identity/v3/users/1f628a14f943417d8f12f6bb2361218a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:17.575489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:17.575997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:17.576101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:17.593236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:17.594442 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:17.637691 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:17.655519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1424/2849] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:17.665828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:17.667266 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:17.694270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:17.727772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8fc3-0a36-46ca-b1c4-427ccb8e6cc8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:17.731479 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:17.732122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:17.732409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:17.732718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:17.741794 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:17.743257 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:17.752957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1426/2850] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:18.027755 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.029735 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:18.042432 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3b152-c43f-4d5d-a788-ab5edeba3bc3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:18.047083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:18.047940 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.048225 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.048500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] PATH_INFO: `/v3/projects/4ef6cb471e6b4f13b23dac9194663e8e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.049717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef6cb471e6b4f13b23dac9194663e8e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:18.051628 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:18.098464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef6cb471e6b4f13b23dac9194663e8e', 'request_id': 'req-857b2075-604f-41f8-ac56-0ac9fcea003a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.104588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef6cb471e6b4f13b23dac9194663e8e', 'request_id': 'req-857b2075-604f-41f8-ac56-0ac9fcea003a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.111867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef6cb471e6b4f13b23dac9194663e8e', 'request_id': 'req-857b2075-604f-41f8-ac56-0ac9fcea003a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.122984 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] The token cache is being invalidate because project 4ef6cb471e6b4f13b23dac9194663e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:18.123273 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.123913 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.125470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1425/2851] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:18 2022] DELETE /identity/v3/projects/4ef6cb471e6b4f13b23dac9194663e8e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:18.129412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.139676 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15dfb2-a1ad-49f5-a657-f4bd8370944f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:18.195873 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:18.196770 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.197062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.197343 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] PATH_INFO: `/v3/users/e6f7a2f7c33049f28f2a65d227588f29` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.198536 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f7a2f7c33049f28f2a65d227588f29)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:18.200470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:18.246712 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f7a2f7c33049f28f2a65d227588f29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.253496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f7a2f7c33049f28f2a65d227588f29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.254144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f7a2f7c33049f28f2a65d227588f29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.254743 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f7a2f7c33049f28f2a65d227588f29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.267023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b3f8702c-aac7-46e2-b104-1eaf40f7e987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f7a2f7c33049f28f2a65d227588f29'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.279900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1427/2852] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:18 2022] DELETE /identity/v3/users/e6f7a2f7c33049f28f2a65d227588f29 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:18.551850 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.553485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:18.566267 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2075-604f-41f8-ac56-0ac9fcea003a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:18.627998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:18.628818 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.629111 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.629392 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] PATH_INFO: `/v3/projects/140ea59124494640811747bf9d6c1518` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.630621 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=140ea59124494640811747bf9d6c1518)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:18.632537 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:18.679723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140ea59124494640811747bf9d6c1518', 'request_id': 'req-35aac913-e07f-4b26-855c-ea184020d0d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.685864 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140ea59124494640811747bf9d6c1518', 'request_id': 'req-35aac913-e07f-4b26-855c-ea184020d0d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.693307 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140ea59124494640811747bf9d6c1518', 'request_id': 'req-35aac913-e07f-4b26-855c-ea184020d0d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.700391 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:18.704748 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] The token cache is being invalidate because project 140ea59124494640811747bf9d6c1518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:18.705211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.706347 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-35aac913-e07f-4b26-855c-ea184020d0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:18.708008 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1426/2853] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:18 2022] DELETE /identity/v3/projects/140ea59124494640811747bf9d6c1518 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:18.713893 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.714352 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.714436 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.735374 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:18.737122 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:18.785935 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:18.811965 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1428/2854] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:18.821401 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.821688 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.821966 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.841188 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:18.842394 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:18.850505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1427/2855] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:18.854729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.864424 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f90766-b715-4bbb-ac73-5164c1d78709 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:18.869056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:18.869706 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.869994 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.870275 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.871011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:45:18.871320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:18.872975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:18.884879 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1429/2856] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:45:18.889036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.898996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bb591-bd7f-4477-8bd6-6d4cdb92e60f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:18.903596 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:18.904236 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:18.904518 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:18.904823 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:18.905507 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:18.907416 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:18.932318 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1428/2857] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:18 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:18.936431 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:18.947383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ffde-6c1b-4d66-9aaa-21f19b0ac6d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.005818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.006782 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.007076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.007359 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.008049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.010037 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.031737 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1430/2858] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:18 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:19.035730 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.047030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdd0bd-5f23-48a8-98f7-c68b32ab7def admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.051617 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.052263 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.052548 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.052902 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.054272 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.055858 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.066153 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1429/2859] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:19.070108 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.080733 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c995ad-4cd2-46aa-86a3-015f9c2d32f4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.085566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.086245 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.086530 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.086811 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.087560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.089495 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d782a6-8047-45cc-8030-9595b66cd013 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.101244 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1431/2860] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:19.105459 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.116135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0861-ab33-41fa-ac86-dbceaf24e1a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.120996 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.122050 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.122425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.122723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] PATH_INFO: `/v3/projects/a7b8e00f92e6427abf5057fcbb8b66c9/users/74393e5dbb064ea890ce5c65a9cce43f/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.137418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b8e00f92e6427abf5057fcbb8b66c9, user_id=74393e5dbb064ea890ce5c65a9cce43f, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.139998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.152997 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1430/2861] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:19 2022] PUT /identity/v3/projects/a7b8e00f92e6427abf5057fcbb8b66c9/users/74393e5dbb064ea890ce5c65a9cce43f/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:19.163808 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.164137 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.164443 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.191026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] MFA Rules not processed for user `74393e5dbb064ea890ce5c65a9cce43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:19.192234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:19.256145 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1432/2862] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:19.260573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.322181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc1a9-41da-4e0e-b26e-df968168328d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.326772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.327418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.327701 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.327979 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.328713 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.330579 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.357298 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1431/2863] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:19 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:19.361274 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.371901 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5931c-4278-407a-aa1a-00dcb11045f4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.409256 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:19.429085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.429758 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.430068 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.430329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.431026 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.432995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.452261 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1433/2864] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:19 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:19.456181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.467033 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515838-d1e1-414a-86ed-933c9d2c0d8e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.471562 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.472211 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.472497 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.472809 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.473525 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.475717 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.486076 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1432/2865] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:19.490034 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.500189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a31a7-737f-4e26-9343-f9b1f529b149 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.505319 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.506324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.506618 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.506903 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] PATH_INFO: `/v3/projects/830f80ff7f984f759353d0634f3b2ca3/users/4dd6710f03224156872d0e2cecdccada/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.522583 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=830f80ff7f984f759353d0634f3b2ca3, user_id=4dd6710f03224156872d0e2cecdccada, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.525302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.537544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1434/2866] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:19 2022] PUT /identity/v3/projects/830f80ff7f984f759353d0634f3b2ca3/users/4dd6710f03224156872d0e2cecdccada/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:19.541593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.551370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952ecf-daed-418d-aa7f-c0c4cdb610cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.608460 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.609176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.609461 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.609738 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.610467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.612086 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08d990-75df-45e6-b615-0aec6e261c0d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.623268 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1433/2867] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:19.627722 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.638062 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe284f-6059-4758-ac73-5c0721004e8f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.642709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.643575 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.643859 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.644133 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] PATH_INFO: `/v3/projects/830f80ff7f984f759353d0634f3b2ca3/users/4dd6710f03224156872d0e2cecdccada/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.646178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830f80ff7f984f759353d0634f3b2ca3, user_id=4dd6710f03224156872d0e2cecdccada, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.649110 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.659190 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1435/2868] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:19 2022] PUT /identity/v3/projects/830f80ff7f984f759353d0634f3b2ca3/users/4dd6710f03224156872d0e2cecdccada/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:19.669723 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.670190 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.670288 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.683369 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:19.698030 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] MFA Rules not processed for user `4dd6710f03224156872d0e2cecdccada`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:19.699235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:19.769773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1434/2869] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 22:45:19.780351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:19.782086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:19.839005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d439a1-2657-4d47-8aa0-572c936766b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:19.842607 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:19.843509 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:19.843797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:19.844072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:19.853582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:19.855027 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27fc5d-1cb2-452d-90c7-3c7cbc87a9f8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:19.864199 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1436/2870] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:20.218441 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:20.280087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644f06f-bcbe-408b-b842-29ab270d13ba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:20.284484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:20.285376 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:20.286181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:20.286181 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] PATH_INFO: `/v3/users/74393e5dbb064ea890ce5c65a9cce43f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:20.287276 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74393e5dbb064ea890ce5c65a9cce43f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:20.289001 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:20.334278 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74393e5dbb064ea890ce5c65a9cce43f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.341069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74393e5dbb064ea890ce5c65a9cce43f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.341680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74393e5dbb064ea890ce5c65a9cce43f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.348510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74393e5dbb064ea890ce5c65a9cce43f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.360658 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74393e5dbb064ea890ce5c65a9cce43f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.367101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1435/2871] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:20 2022] DELETE /identity/v3/users/74393e5dbb064ea890ce5c65a9cce43f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:20.371582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:20.371874 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:20.372154 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:20.389719 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:20.390902 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:20.448671 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1437/2872] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:20.458218 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:20.459783 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:20.473581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:20.524992 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56acc7-0234-4187-a8ab-b88498afe8a3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:20.528588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:20.529264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:20.529549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:20.529829 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:20.540015 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:20.541510 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:20.550832 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1436/2873] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:20.818100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:20.819681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:20.831784 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305e546-5d42-4907-a8f2-d88cab58fe3e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:20.837088 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:20.837952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:20.838234 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:20.838772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] PATH_INFO: `/v3/projects/a7b8e00f92e6427abf5057fcbb8b66c9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:20.839949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b8e00f92e6427abf5057fcbb8b66c9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:20.841905 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:20.876243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:20.891178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b8e00f92e6427abf5057fcbb8b66c9', 'request_id': 'req-17c08889-b113-4158-a09b-95fd42618104'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.897815 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b8e00f92e6427abf5057fcbb8b66c9', 'request_id': 'req-17c08889-b113-4158-a09b-95fd42618104'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.906463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b8e00f92e6427abf5057fcbb8b66c9', 'request_id': 'req-17c08889-b113-4158-a09b-95fd42618104'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.917232 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] The token cache is being invalidate because project a7b8e00f92e6427abf5057fcbb8b66c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:20.917513 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.918415 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:20.920084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1438/2874] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:20 2022] DELETE /identity/v3/projects/a7b8e00f92e6427abf5057fcbb8b66c9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:20.924101 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:20.933943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8abf1-40dd-4592-b8f8-95022657e082 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:20.990004 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:20.990851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:20.991134 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:20.991454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] PATH_INFO: `/v3/users/4dd6710f03224156872d0e2cecdccada` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:20.992676 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd6710f03224156872d0e2cecdccada)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:20.994588 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.042806 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd6710f03224156872d0e2cecdccada'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.050026 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd6710f03224156872d0e2cecdccada'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.050667 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd6710f03224156872d0e2cecdccada'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.058043 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd6710f03224156872d0e2cecdccada'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.070219 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a3f294ce-08d5-46b2-be27-67cc1d74ad38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd6710f03224156872d0e2cecdccada'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.076095 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1437/2875] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:20 2022] DELETE /identity/v3/users/4dd6710f03224156872d0e2cecdccada => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:45:21.355056 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.356470 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:21.370328 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08889-b113-4158-a09b-95fd42618104 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.427381 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.428461 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.428919 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.429206 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] PATH_INFO: `/v3/projects/830f80ff7f984f759353d0634f3b2ca3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.430410 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830f80ff7f984f759353d0634f3b2ca3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.432383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.479865 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830f80ff7f984f759353d0634f3b2ca3', 'request_id': 'req-64bc2f04-b84c-467e-89d4-3c9738433516'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.485992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830f80ff7f984f759353d0634f3b2ca3', 'request_id': 'req-64bc2f04-b84c-467e-89d4-3c9738433516'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.493005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830f80ff7f984f759353d0634f3b2ca3', 'request_id': 'req-64bc2f04-b84c-467e-89d4-3c9738433516'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.503069 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] The token cache is being invalidate because project 830f80ff7f984f759353d0634f3b2ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:21.503357 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.503973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-64bc2f04-b84c-467e-89d4-3c9738433516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:21.505544 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1439/2876] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:21 2022] DELETE /identity/v3/projects/830f80ff7f984f759353d0634f3b2ca3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:21.510446 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.510742 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.511027 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.530227 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:21.531358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:21.560798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:21.592947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1438/2877] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:21.601746 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.602036 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.602320 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.621959 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:21.623164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:21.631826 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1440/2878] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:21.635965 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.645725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88a23e-ab76-46d3-83c1-c731da2e6967 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.650867 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.651522 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.651812 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.652094 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.652866 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:45:21.653182 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.654774 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.666899 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1439/2879] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:45:21.670983 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.680186 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3fbc1-e478-4e4a-8f94-74c70e0b3456 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.684973 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.685664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.685943 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.686225 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.686949 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.688911 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.714308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1441/2880] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:21 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:21.718402 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.729292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90daf-9dcf-409d-a930-9695b2536d92 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.784697 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.785351 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.785640 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.785925 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.786626 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.788770 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.808734 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1440/2881] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:21 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:21.812498 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.823882 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe26f3-541e-42d8-add0-ea29b42735eb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.829727 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.830383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.830664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.830940 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.831662 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.833321 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.844054 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1442/2882] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:21.848235 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.859185 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe393-ca73-40e1-80c4-b2915d8ff65e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.863948 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.864605 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.864916 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.865199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.866433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.868062 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a63a0e-0dfa-4830-aa1a-ef11f1f9abb3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.878079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1441/2883] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:21.882860 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:21.892819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443e88-c6a5-44e0-a262-7254cb5ce42a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:21.897769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:21.898638 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.898920 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.899195 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] PATH_INFO: `/v3/projects/c4b15923716c445a90e852d410e3fa8e/users/2c91250ca7cf464580ca8cd9322b9cca/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.903004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:21.917649 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b15923716c445a90e852d410e3fa8e, user_id=2c91250ca7cf464580ca8cd9322b9cca, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:21.920653 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:21.933921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1443/2884] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:21 2022] PUT /identity/v3/projects/c4b15923716c445a90e852d410e3fa8e/users/2c91250ca7cf464580ca8cd9322b9cca/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:21.944876 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:21.945370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:21.945458 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:21.972025 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] MFA Rules not processed for user `2c91250ca7cf464580ca8cd9322b9cca`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:21.973264 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:22.040270 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1442/2885] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:22.050772 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:22.052489 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:22.113640 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6bc57-b115-4e96-87de-a893ad82b99f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:22.117912 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:22.118560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:22.118875 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:22.119155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:22.127702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:22.129224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789642-c09c-4942-b6ca-638cbc8622d9 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:22.138178 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1444/2886] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:30.529047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:30.562527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:30.596257 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000a3c3-a0a6-4dd1-8847-3174ea7bb187 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:30.601898 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:30.602760 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:30.603047 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:30.603328 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] PATH_INFO: `/v3/users/2c91250ca7cf464580ca8cd9322b9cca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:30.604511 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c91250ca7cf464580ca8cd9322b9cca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:30.606454 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:30.654191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c91250ca7cf464580ca8cd9322b9cca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:30.661390 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c91250ca7cf464580ca8cd9322b9cca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:30.662203 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c91250ca7cf464580ca8cd9322b9cca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:30.669375 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c91250ca7cf464580ca8cd9322b9cca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:30.681708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c91250ca7cf464580ca8cd9322b9cca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:30.687486 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1443/2887] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:30 2022] DELETE /identity/v3/users/2c91250ca7cf464580ca8cd9322b9cca => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:30.692007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:30.692473 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:30.692563 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:30.710390 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:30.711595 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:30.770396 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1445/2888] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:30.780081 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:30.781532 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:30.840464 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f209-6683-4b64-b136-1a8495c935cb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:30.844127 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:30.844808 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:30.845097 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:30.845381 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:30.855485 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:30.856947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e873eb-3bed-4a08-86a5-978009cb3e33 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:30.867069 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1444/2889] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:31.133023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.134566 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:31.150716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb81a1-2945-48d1-bd0c-979ba896c272 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.155718 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.156610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.156980 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.157189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] PATH_INFO: `/v3/projects/c4b15923716c445a90e852d410e3fa8e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.158384 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b15923716c445a90e852d410e3fa8e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.160308 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.210609 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b15923716c445a90e852d410e3fa8e', 'request_id': 'req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:31.217140 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b15923716c445a90e852d410e3fa8e', 'request_id': 'req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:31.224483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b15923716c445a90e852d410e3fa8e', 'request_id': 'req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:31.234975 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] The token cache is being invalidate because project c4b15923716c445a90e852d410e3fa8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:31.235267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:31.235892 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8cbc9cd0-eddb-44c7-8cc5-9968b54bae56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:31.237483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1446/2890] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:31 2022] DELETE /identity/v3/projects/c4b15923716c445a90e852d410e3fa8e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:31.243380 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.243702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.243987 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.262176 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:31.263378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:31.314408 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:31.331017 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1445/2891] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:31.340004 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.340294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.340572 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.359520 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:31.360933 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:31.370827 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1447/2892] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:31.375177 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.385638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091730c3-855d-417b-9b1c-3ebe2ed6168b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.390162 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.390816 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.391106 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.391411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.392115 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:45:31.392424 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.394042 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.405708 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1446/2893] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:45:31.409971 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.416487 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:31.422299 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d34860-1b01-461b-ac91-cfe5abefda6f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.428944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.429828 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.430284 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.430623 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.431522 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.433561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.462952 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1448/2894] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:31.464411 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.474418 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947686bf-fb58-4797-ba58-5020a2d07ac4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.529581 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.530283 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.530448 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.530725 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.531400 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.533329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.552421 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1447/2895] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:31 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:31.557267 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.567556 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353bb22-4e06-4afb-89bb-20d48ac894c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.572957 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.573615 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.573906 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.574187 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.574914 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.576528 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.586956 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1449/2896] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 22:45:31.590975 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.601870 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc59-7d6b-4d94-a9b7-5bf0e31be74f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.606491 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.607140 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.607425 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.607702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.608423 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.610041 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308c1f5-caa8-4f97-b492-88b51e8669e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.620817 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1448/2897] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:31.624913 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.635049 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c020-3c94-476b-ab55-f13392f80542 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.639739 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.640803 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.641090 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.641387 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] PATH_INFO: `/v3/projects/a98f4e28b0f24096a886697c570a6b68/users/f061942fe0aa4483b6374672a94f6afd/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.656249 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f4e28b0f24096a886697c570a6b68, user_id=f061942fe0aa4483b6374672a94f6afd, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.658953 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.672448 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1450/2898] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:31 2022] PUT /identity/v3/projects/a98f4e28b0f24096a886697c570a6b68/users/f061942fe0aa4483b6374672a94f6afd/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:31.683538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.683836 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.684122 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.710549 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] MFA Rules not processed for user `f061942fe0aa4483b6374672a94f6afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:31.711750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:31.778193 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1449/2899] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:31.789710 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:31.791466 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:31.847602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53119244-59b0-44cb-abbd-e85033e38789 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:31.851923 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41cfa205823a4c29abea3001bf086711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:31.852601 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:31.852910 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:31.853189 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:31.861394 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:31.862834 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353db01-5fc9-409e-b65a-305ec4882bc9 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:31.872351 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1451/2900] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 22:45:34.864098 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:34.912752 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:34.928604 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e194b-6159-4c65-9239-b187ba96d1c7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:34.933199 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:34.934070 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:34.934355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:34.934634 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] PATH_INFO: `/v3/users/f061942fe0aa4483b6374672a94f6afd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:34.935843 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f061942fe0aa4483b6374672a94f6afd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:34.937801 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:34.985672 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f061942fe0aa4483b6374672a94f6afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:34.993335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f061942fe0aa4483b6374672a94f6afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:34.994075 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f061942fe0aa4483b6374672a94f6afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.001129 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f061942fe0aa4483b6374672a94f6afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.013587 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f061942fe0aa4483b6374672a94f6afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.021436 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1450/2901] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:45:34 2022] DELETE /identity/v3/users/f061942fe0aa4483b6374672a94f6afd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:35.025900 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.026380 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.026480 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.043693 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:35.044981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:35.103329 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1452/2902] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:35.112947 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.114356 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:35.172393 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe9fe3-b0c7-4977-9df6-ad760d36ee76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.176011 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.176684 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.176977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.177254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.186938 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.188358 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca0b55-4256-4dda-8aad-5fffc17a80ef service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.198495 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1451/2903] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:35.471622 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.473180 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:35.485930 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe156971-ee65-4369-964d-ac39c0a55b23 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.491105 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.491976 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.492263 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.492541 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] PATH_INFO: `/v3/projects/a98f4e28b0f24096a886697c570a6b68` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.493797 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98f4e28b0f24096a886697c570a6b68)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.495730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.542403 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:35.546191 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98f4e28b0f24096a886697c570a6b68', 'request_id': 'req-53b64420-b862-494a-9557-5a0c27e9031e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.554624 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98f4e28b0f24096a886697c570a6b68', 'request_id': 'req-53b64420-b862-494a-9557-5a0c27e9031e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.562063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98f4e28b0f24096a886697c570a6b68', 'request_id': 'req-53b64420-b862-494a-9557-5a0c27e9031e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.572482 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] The token cache is being invalidate because project a98f4e28b0f24096a886697c570a6b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:45:35.572813 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.573460 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-53b64420-b862-494a-9557-5a0c27e9031e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:45:35.575000 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1453/2904] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:45:35 2022] DELETE /identity/v3/projects/a98f4e28b0f24096a886697c570a6b68 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:35.579995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.580292 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.580575 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.599034 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:35.600797 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:35.660032 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1452/2905] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:35.668552 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.668883 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.669170 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.691451 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:35.693602 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:35.703086 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1454/2906] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:35.707377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.718260 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e0e2-2779-4c6f-a355-1f80706881f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.722781 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.723431 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.723715 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.723995 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.725389 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 22:45:35.725638 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.727230 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.729544 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:35.741124 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1453/2907] 10.222.0.61 () {58 vars in 1152 bytes} [Thu May 19 22:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 22:45:35.745192 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.754578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7603a14-ac8d-401a-8b7c-486aef241aed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.759363 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.760007 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.760292 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.760568 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.761298 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.763214 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.789236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1455/2908] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:35 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:35.793494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.803505 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35774658-91f7-4002-aa29-1314489e3741 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.860061 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.860754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.861046 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.861329 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.862023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.863943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.885346 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1454/2909] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:35 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:35.889412 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.900296 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f1472-09ab-41c5-bb46-3dffa2228b25 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.905134 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.905799 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.906164 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.906449 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.907203 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.908862 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.919831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1456/2910] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:35.923679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.933943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040ba2c-70da-42ff-a414-6528869ec879 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.940533 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.941229 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.941515 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.941798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.942527 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.944322 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f065bf6-e94b-42e9-a367-01238b907328 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:35.954319 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1455/2911] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:35.959059 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:35.969318 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81de6-f8ce-474b-b89d-3009a1dff852 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:35.973819 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:35.975366 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:35.975650 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:35.975921 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] PATH_INFO: `/v3/projects/7afedea5cb2a476d8960f5c9d2aeba03/users/63f199e3dafc40b7aba09c3e71855468/roles/c2ae6dbe1e314175bcda8fcf0e179597` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:35.989847 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afedea5cb2a476d8960f5c9d2aeba03, user_id=63f199e3dafc40b7aba09c3e71855468, role_id=c2ae6dbe1e314175bcda8fcf0e179597)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:35.992483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:36.005944 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1457/2912] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:35 2022] PUT /identity/v3/projects/7afedea5cb2a476d8960f5c9d2aeba03/users/63f199e3dafc40b7aba09c3e71855468/roles/c2ae6dbe1e314175bcda8fcf0e179597 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:36.011036 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c085-32de-4116-878a-52a6d34ba595 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:36.011433 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c085-32de-4116-878a-52a6d34ba595 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:36.011750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c085-32de-4116-878a-52a6d34ba595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:36.032265 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d3c5c085-32de-4116-878a-52a6d34ba595 None None] MFA Rules not processed for user `d32395c520864a849d965edd5ff1c3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:36.033503 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3c5c085-32de-4116-878a-52a6d34ba595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:36.095722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1456/2913] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:36.105324 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:36.107087 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:36.163155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888062-62c9-429c-89ac-e3cf7a56c03f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:36.166781 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:36.167420 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:36.167702 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:36.167981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:36.176888 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:36.178569 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:36.188176 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1458/2914] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:39.954933 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:39.955451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:39.955451 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:39.983271 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] MFA Rules not processed for user `63f199e3dafc40b7aba09c3e71855468`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:39.984487 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:40.052519 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1457/2915] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:40.056981 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.066972 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eb352-e871-41e0-8ea9-65561e7619e9 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.072279 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.073084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.073416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.073730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.074508 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.076610 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.104755 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1459/2916] 10.222.0.61 () {60 vars in 1152 bytes} [Thu May 19 22:45:40 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:40.108863 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.117719 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:40.122784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839661-2253-4cd6-a142-3b8e885e5cbc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.186083 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.186790 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.187079 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.187370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.188088 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.190139 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.210991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1458/2917] 10.222.0.61 () {60 vars in 1143 bytes} [Thu May 19 22:45:40 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 22:45:40.215144 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.226101 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b9d99-c292-4c2a-87c9-6955b407a868 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.230630 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.231302 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.231555 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.231831 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.232771 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.234383 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.246078 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1460/2918] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:40.250249 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.261484 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10888d-1657-4eff-a0d2-51d272ebed52 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.266313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.267209 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.267494 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.267773 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] PATH_INFO: `/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6/users/8f994fa512854413a10c9faec61e4e24/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.293754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0f0132fc3e47e89a1e0a7aa6e282c6, user_id=8f994fa512854413a10c9faec61e4e24, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.296784 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.319674 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1459/2919] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:40 2022] PUT /identity/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6/users/8f994fa512854413a10c9faec61e4e24/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:45:40.323514 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.333416 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8723-792d-48f5-8741-df58b795d169 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.381917 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:40.403084 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.403773 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.404063 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.404344 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.405115 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.406752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d02ba4-3d5b-44e6-b038-6c504fbae9b9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.424429 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1461/2920] 10.222.0.61 () {58 vars in 1121 bytes} [Thu May 19 22:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 22:45:40.426143 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:40.436976 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4871-eb01-468f-a5c6-430799852d35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:40.441608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:40.442500 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:40.442785 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:40.443096 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] PATH_INFO: `/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6/users/8f994fa512854413a10c9faec61e4e24/roles/6a9f236c49c24283afb390181f99e500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:40.445087 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0f0132fc3e47e89a1e0a7aa6e282c6, user_id=8f994fa512854413a10c9faec61e4e24, role_id=6a9f236c49c24283afb390181f99e500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:40.447793 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:40.459922 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1460/2921] 10.222.0.61 () {60 vars in 1482 bytes} [Thu May 19 22:45:40 2022] PUT /identity/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6/users/8f994fa512854413a10c9faec61e4e24/roles/6a9f236c49c24283afb390181f99e500 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:45:45.166715 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfb64-b673-4394-b204-041544031a48 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:45.167725 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfb64-b673-4394-b204-041544031a48 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:45.168231 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cfb64-b673-4394-b204-041544031a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:45.203023 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fb3cfb64-b673-4394-b204-041544031a48 None None] MFA Rules not processed for user `8f994fa512854413a10c9faec61e4e24`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:45:45.204247 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fb3cfb64-b673-4394-b204-041544031a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:45.276200 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1462/2922] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:45:45.292330 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:45:45.294240 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:45:45.384130 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f47ce-4733-4c13-9eb2-d6db4111eca3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:45:45.392023 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:45:45.392912 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:45:45.393195 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:45:45.393472 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:45:45.465955 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:45.470927 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:45:45.473118 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:45:45.485310 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1461/2923] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:45:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:45:55.272995 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=45 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:45:55.486680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:46:02.741224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:46:02.741723 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:46:02.741833 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:46:02.758818 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] MFA Rules not processed for user `63f199e3dafc40b7aba09c3e71855468`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:46:02.760002 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:46:02.766738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1463/2924] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:46:12.753641 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:46:18.846172 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:46:18.908084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d856b60-a6ac-4694-a19f-76317799997b service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:46:18.911683 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:46:18.912332 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:46:18.912639 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:46:18.912932 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:46:18.922409 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:46:18.923849 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:46:18.933854 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1462/2925] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:46:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:46:28.930316 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=29 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:48:16.597502 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:16.599011 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:16.639165 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a8b9-ec57-4381-86de-905ab0399cc1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:16.642779 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:16.643421 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:16.643709 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:16.643992 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:16.645769 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:16.655604 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:16.657085 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:16.665668 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1464/2926] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:18.648224 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:18.674805 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7be81-6be0-4315-938e-6848fb6ce3f1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:18.678377 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:18.679012 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:18.679297 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:18.679573 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:18.681470 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:18.694872 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:18.696943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:18.707988 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1463/2927] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:19.884752 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:19.893581 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d26d3-f239-464b-8bfb-b4f075631cb3 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:19.897558 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:19.898196 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:19.898483 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:19.898762 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:19.901066 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:19.914475 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:19.916806 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c780639-9967-45c9-b214-f59a71ddddae service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:19.928028 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1465/2928] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:21.208798 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:21.209745 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:21.226085 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee03ce-b9fc-47bc-bf84-85c3bfba14bc service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:21.232977 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:21.233650 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:21.233943 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:21.234241 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:21.236285 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:21.252191 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:21.253991 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:21.266538 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1464/2929] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:22.206985 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:22.207439 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:22.207578 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:22.225557 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] MFA Rules not processed for user `63f199e3dafc40b7aba09c3e71855468`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:48:22.226750 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:22.234413 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1466/2930] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:48:22.543687 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:22.553028 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef829a6-f37b-45ef-b1e3-3497e191dbf3 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:22.557627 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:22.558261 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:22.558547 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:22.558826 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:22.560593 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:22.570022 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:22.571583 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:22.579598 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1465/2931] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:23.034076 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:23.035474 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:23.110236 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc08720-6017-4587-8900-5b7556ef4d86 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:23.115948 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:23.116603 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:23.116924 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:23.117207 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:23.119526 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:23.130891 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:23.132496 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:23.143210 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1467/2932] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:32.576029 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=24 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:48:33.139005 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=58 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:48:34.899733 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:34.915084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cb9eb-c627-4062-8eac-107503173bd5 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:34.920434 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508245-1564-4196-baf3-42b1167841cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:34.921135 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508245-1564-4196-baf3-42b1167841cb service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:34.921428 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508245-1564-4196-baf3-42b1167841cb service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:34.921709 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508245-1564-4196-baf3-42b1167841cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:34.923529 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27508245-1564-4196-baf3-42b1167841cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:34.936607 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508245-1564-4196-baf3-42b1167841cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:34.938238 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508245-1564-4196-baf3-42b1167841cb service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:34.950467 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1466/2933] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:40.234757 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:48:40.247211 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945333b1-8179-4e6d-b054-9e02af392f52 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:48:40.252692 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6fbc800c444f2aa44f2c33c40149f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:48:40.253341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:48:40.253642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:48:40.253939 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:48:40.255730 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:48:40.268490 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:48:40.270175 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806691c9-eecf-4f32-a57b-c50117e89e2c service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:48:40.281795 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1468/2934] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:48:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:48:44.944671 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:48:50.277075 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:50:14.912937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508245-1564-4196-baf3-42b1167841cb service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:50:14.914337 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27508245-1564-4196-baf3-42b1167841cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:50:14.935212 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508245-1564-4196-baf3-42b1167841cb service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:50:14.940269 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:50:14.940998 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:50:14.941313 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:50:14.941594 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:50:14.943378 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:50:14.955149 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:50:14.956750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ea107-d2ee-4479-b601-258f262b601c service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:50:14.966989 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1467/2935] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:50:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:50:15.613740 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:50:15.614243 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:50:15.614333 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:50:15.631409 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] MFA Rules not processed for user `63f199e3dafc40b7aba09c3e71855468`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:50:15.632582 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:50:15.639749 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1469/2936] 10.222.0.61 () {58 vars in 957 bytes} [Thu May 19 22:50:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:50:24.962363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:50:25.626241 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:11.818699 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:11.819197 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:11.819296 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:11.876137 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] MFA Rules not processed for user `5cfd4c34ecd84abe8310817755ae62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 22:52:11.877407 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:52:11.884355 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1468/2937] 10.222.0.61 () {58 vars in 1061 bytes} [Thu May 19 22:52:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 22:52:11.890121 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:11.900072 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdfa17-f2da-4219-8b75-a6d6e0b01c7b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:11.903642 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:11.904277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:11.904560 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:11.904872 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:11.913052 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:11.914517 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:11.923963 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1470/2938] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:52:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:52:21.873772 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=17 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:21.920014 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:25.412374 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:25.420586 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3832ab-5ec4-4c3a-99e9-305a13a73a8c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:25.424360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cfd4c34ecd84abe8310817755ae62fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:25.425038 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:25.425321 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:25.425597 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:25.499254 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:25.500722 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:25.520147 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1469/2939] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:52:28.533852 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:28.547591 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc401-ccf5-4b56-ad9d-0181f32e8c91 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:28.557463 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:28.558632 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:28.559152 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:28.559664 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:28.563294 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:52:28.586322 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:28.590205 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:28.609398 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1471/2940] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:52:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:52:31.589155 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:31.607326 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f731680-4f97-480a-9367-875f94954b4a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:31.612489 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:31.613399 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:31.613682 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:31.613961 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] PATH_INFO: `/v3/users/63f199e3dafc40b7aba09c3e71855468` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:31.615156 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f199e3dafc40b7aba09c3e71855468)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:31.617091 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:31.672692 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f199e3dafc40b7aba09c3e71855468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:31.682563 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f199e3dafc40b7aba09c3e71855468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:31.683215 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f199e3dafc40b7aba09c3e71855468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:31.691880 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f199e3dafc40b7aba09c3e71855468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:31.711768 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f199e3dafc40b7aba09c3e71855468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:31.719370 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1470/2941] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:52:31 2022] DELETE /identity/v3/users/63f199e3dafc40b7aba09c3e71855468 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:52:31.907734 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:31.919166 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c7a5-516d-4786-8c63-eb409b46fdac service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:31.979305 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:31.980277 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:31.980561 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:31.980868 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] PATH_INFO: `/v3/projects/7afedea5cb2a476d8960f5c9d2aeba03` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:31.982047 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afedea5cb2a476d8960f5c9d2aeba03)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:31.983982 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:32.031288 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afedea5cb2a476d8960f5c9d2aeba03', 'request_id': 'req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.037820 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afedea5cb2a476d8960f5c9d2aeba03', 'request_id': 'req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.045729 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afedea5cb2a476d8960f5c9d2aeba03', 'request_id': 'req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.069270 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] The token cache is being invalidate because project 7afedea5cb2a476d8960f5c9d2aeba03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:52:32.069564 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.070242 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.071789 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1472/2942] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:52:31 2022] DELETE /identity/v3/projects/7afedea5cb2a476d8960f5c9d2aeba03 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:52:32.075680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:32.077123 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:52:32.090498 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e7ff-c0a5-4bcc-ae47-2d49f289fb1d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:32.140084 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:32.153110 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:32.153917 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:32.154200 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:32.154475 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] PATH_INFO: `/v3/users/8f994fa512854413a10c9faec61e4e24` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:32.155739 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f994fa512854413a10c9faec61e4e24)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:32.157680 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:32.204520 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f994fa512854413a10c9faec61e4e24'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.211851 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f994fa512854413a10c9faec61e4e24'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.212608 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f994fa512854413a10c9faec61e4e24'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.220092 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f994fa512854413a10c9faec61e4e24'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.232750 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f994fa512854413a10c9faec61e4e24'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.238874 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1471/2943] 10.222.0.61 () {58 vars in 1223 bytes} [Thu May 19 22:52:32 2022] DELETE /identity/v3/users/8f994fa512854413a10c9faec61e4e24 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 22:52:32.407716 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:52:32.409246 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:52:32.423074 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e007-ea95-4aea-9438-58bf5bbfc9d8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:52:32.488100 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd32395c520864a849d965edd5ff1c3ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906e152e20d14899a1c568a7ddf7bb54', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:52:32.488999 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:52:32.489281 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:52:32.489559 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] PATH_INFO: `/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:52:32.490717 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0f0132fc3e47e89a1e0a7aa6e282c6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:52:32.492690 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:52:32.500627 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:32.539155 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0f0132fc3e47e89a1e0a7aa6e282c6', 'request_id': 'req-77aeb0f8-2cd3-4e29-be1f-a962d7483847'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.546112 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0f0132fc3e47e89a1e0a7aa6e282c6', 'request_id': 'req-77aeb0f8-2cd3-4e29-be1f-a962d7483847'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.553341 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0f0132fc3e47e89a1e0a7aa6e282c6', 'request_id': 'req-77aeb0f8-2cd3-4e29-be1f-a962d7483847'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.563738 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] The token cache is being invalidate because project 8f0f0132fc3e47e89a1e0a7aa6e282c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 22:52:32.564031 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.564681 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-77aeb0f8-2cd3-4e29-be1f-a962d7483847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 22:52:32.566224 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1473/2944] 10.222.0.61 () {58 vars in 1232 bytes} [Thu May 19 22:52:32 2022] DELETE /identity/v3/projects/8f0f0132fc3e47e89a1e0a7aa6e282c6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 22:52:42.235360 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:52:42.560776 dv-cinder-842633-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:55:18.867335 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:55:18.868710 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:55:18.999053 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd27c8-1e08-4cc4-ab42-7c3a6aa57170 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:55:19.060702 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb31582d6fc14a7f9ec20d107c445390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df7a31e797b24706a0b85db1ee03cc75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:55:19.061363 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:55:19.061653 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:55:19.061937 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:55:19.063754 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:55:19.074941 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:55:19.076694 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca291fd-283d-4775-ae64-067d29b6d828 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:55:19.086679 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1472/2945] 10.222.0.61 () {60 vars in 1365 bytes} [Thu May 19 22:55:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:55:29.082384 dv-cinder-842633-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}